Created by three guys who love BSD, we cover the latest news and have an extensive series of tutorials, as well as interviews with various people from all areas of the BSD community. It also serves as a platform for support and questions. We love and advocate FreeBSD, OpenBSD, NetBSD, DragonFlyBSD and TrueOS. Our show aims to be helpful and informative for new users that want to learn about them, but still be entertaining for the people who are already pros. The show airs on Wednesdays at 2:00PM (US Eastern time) and the edited version is usually up the following day.
Similar Podcasts
Thinking Elixir Podcast
The Thinking Elixir podcast is a weekly show where we talk about the Elixir programming language and the community around it. We cover news and interview guests to learn more about projects and developments in the community.
The Cynical Developer
A UK based Technology and Software Developer Podcast that helps you to improve your development knowledge and career,
through explaining the latest and greatest in development technology and providing you with what you need to succeed as a developer.
Elixir Outlaws
Elixir Outlaws is an informal discussion about interesting things happening in Elixir. Our goal is to capture the spirit of a conference hallway discussion in a podcast.
Episode 268: Netcat Demystified | BSD Now 268
6 metrics for zpool performance, 2FA with ssh on OpenBSD, ZFS maintaining file type information in dirs, everything old is new again, netcat demystified, and more. ##Headlines Six Metrics for Measuring ZFS Pool Performance Part 1 The layout of a ZFS storage pool has a significant impact on system performance under various workloads. Given the importance of picking the right configuration for your workload and the fact that making changes to an in-use ZFS pool is far from trivial, it is important for an administrator to understand the mechanics of pool performance when designing a storage system. To quantify pool performance, we will consider six primary metrics: Read I/O operations per second (IOPS) Write IOPS Streaming read speed Streaming write speed Storage space efficiency (usable capacity after parity versus total raw capacity) Fault tolerance (maximum number of drives that can fail before data loss) For the sake of comparison, we’ll use an example system with 12 drives, each one sized at 6TB, and say that each drive does 100MB/s streaming reads and writes and can do 250 read and write IOPS. We will visualize how the data is spread across the drives by writing 12 multi-colored blocks, shown below. The blocks are written to the pool starting with the brown block on the left (number one), and working our way to the pink block on the right (number 12). Note that when we calculate data rates and IOPS values for the example system, they are only approximations. Many other factors can impact pool access speeds for better (compression, caching) or worse (poor CPU performance, not enough memory). There is no single configuration that maximizes all six metrics. Like so many things in life, our objective is to find an appropriate balance of the metrics to match a target workload. For example, a cold-storage backup system will likely want a pool configuration that emphasizes usable storage space and fault tolerance over the other data-rate focused metrics. Let’s start with a quick review of ZFS storage pools before diving into specific configuration options. ZFS storage pools are comprised of one or more virtual devices, or vdevs. Each vdev is comprised of one or more storage providers, typically physical hard disks. All disk-level redundancy is configured at the vdev level. That is, the RAID layout is set on each vdev as opposed to on the storage pool. Data written to the storage pool is then striped across all the vdevs. Because pool data is striped across the vdevs, the loss of any one vdev means total pool failure. This is perhaps the single most important fact to keep in mind when designing a ZFS storage system. We will circle back to this point in the next post, but keep it in mind as we go through the vdev configuration options. Because storage pools are made up of one or more vdevs with the pool data striped over the top, we’ll take a look at pool configuration in terms of various vdev configurations. There are three basic vdev configurations: striping, mirroring, and RAIDZ (which itself has three different varieties). The first section will cover striped and mirrored vdevs in this post; the second post will cover RAIDZ and some example scenarios. A striped vdev is the simplest configuration. Each vdev consists of a single disk with no redundancy. When several of these single-disk, striped vdevs are combined into a single storage pool, the total usable storage space would be the sum of all the drives. When you write data to a pool made of striped vdevs, the data is broken into small chunks called “blocks” and distributed across all the disks in the pool. The blocks are written in “round-robin” sequence, meaning after all the disks receive one row of blocks, called a stripe, it loops back around and writes another stripe under the first. A striped pool has excellent performance and storage space efficiency, but absolutely zero fault tolerance. If even a single drive in the pool fails, the entire pool will fail and all data stored on that pool will be lost. The excellent performance of a striped pool comes from the fact that all of the disks can work independently for all read and write operations. If you have a bunch of small read or write operations (IOPS), each disk can work independently to fetch the next block. For streaming reads and writes, each disk can fetch the next block in line synchronized with its neighbors. For example, if a given disk is fetching block n, its neighbor to the left can be fetching block n-1, and its neighbor to the right can be fetching block n+1. Therefore, the speed of all read and write operations as well as the quantity of read and write operations (IOPS) on a striped pool will scale with the number of vdevs. Note here that I said the speeds and IOPS scale with the number of vdevs rather than the number of drives; there’s a reason for this and we’ll cover it in the next post when we discuss RAID-Z. Here’s a summary of the total pool performance (where N is the number of disks in the pool): N-wide striped: Read IOPS: N * Read IOPS of a single drive Write IOPS: N * Write IOPS of a single drive Streaming read speed: N * Streaming read speed of a single drive Streaming write speed: N * Streaming write speed of a single drive Storage space efficiency: 100% Fault tolerance: None! Let’s apply this to our example system, configured with a 12-wide striped pool: 12-wide striped: Read IOPS: 3000 Write IOPS: 3000 Streaming read speed: 1200 MB/s Streaming write speed: 1200 MB/s Storage space efficiency: 72 TB Fault tolerance: None! Below is a visual depiction of our 12 rainbow blocks written to this pool configuration: The blocks are simply striped across the 12 disks in the pool. The LBA column on the left stands for “Logical Block Address”. If we treat each disk as a column in an array, each LBA would be a row. It’s also easy to see that if any single disk fails, we would be missing a color in the rainbow and our data would be incomplete. While this configuration has fantastic read and write speeds and can handle a ton of IOPS, the data stored on the pool is very vulnerable. This configuration is not recommended unless you’re comfortable losing all of your pool’s data whenever any single drive fails. A mirrored vdev consists of two or more disks. A mirrored vdev stores an exact copy of all the data written to it on each one of its drives. Traditional RAID-1 mirrors usually only support two drive mirrors, but ZFS allows for more drives per mirror to increase redundancy and fault tolerance. All disks in a mirrored vdev have to fail for the vdev, and thus the whole pool, to fail. Total storage space will be equal to the size of a single drive in the vdev. If you’re using mismatched drive sizes in your mirrors, the total size will be that of the smallest drive in the mirror. Streaming read speeds and read IOPS on a mirrored vdev will be faster than write speeds and IOPS. When reading from a mirrored vdev, the drives can “divide and conquer” the operations, similar to what we saw above in the striped pool. This is because each drive in the mirror has an identical copy of the data. For write operations, all of the drives need to write a copy of the data, so the mirrored vdev will be limited to the streaming write speed and IOPS of a single disk. Here’s a summary: N-way mirror: Read IOPS: N * Read IOPS of a single drive Write IOPS: Write IOPS of a single drive Streaming read speed: N * Streaming read speed of a single drive Streaming write speed: Streaming write speed of a single drive Storage space efficiency: 50% for 2-way, 33% for 3-way, 25% for 4-way, etc. [(N-1)/N] Fault tolerance: 1 disk per vdev for 2-way, 2 for 3-way, 3 for 4-way, etc. [N-1] For our first example configuration, let’s do something ridiculous and create a 12-way mirror. ZFS supports this kind of thing, but your management probably will not. 1x 12-way mirror: Read IOPS: 3000 Write IOPS: 250 Streaming read speed: 1200 MB/s Streaming write speed: 100 MB/s Storage space efficiency: 8.3% (6 TB) Fault tolerance: 11 As we can clearly see from the diagram, every single disk in the vdev gets a full copy of our rainbow data. The chainlink icons between the disk labels in the column headers indicate the disks are part of a single vdev. We can lose up to 11 disks in this vdev and still have a complete rainbow. Of course, the data takes up far too much room on the pool, occupying a full 12 LBAs in the data array. Obviously, this is far from the best use of 12 drives. Let’s do something a little more practical and configure the pool with the ZFS equivalent of RAID-10. We’ll configure six 2-way mirror vdevs. ZFS will stripe the data across all 6 of the vdevs. We can use the work we did in the striped vdev section to determine how the pool as a whole will behave. Let’s first calculate the performance per vdev, then we can work on the full pool: 1x 2-way mirror: Read IOPS: 500 Write IOPS: 250 Streaming read speed: 200 MB/s Streaming write speed: 100 MB/s Storage space efficiency: 50% (6 TB) Fault tolerance: 1 Now we can pretend we have 6 drives with the performance statistics listed above and run them through our striped vdev performance calculator to get the total pool’s performance: 6x 2-way mirror: Read IOPS: 3000 Write IOPS: 1500 Streaming read speed: 3000 MB/s Streaming write speed: 1500 MB/s Storage space efficiency: 50% (36 TB) Fault tolerance: 1 per vdev, 6 total Again, we will examine the configuration from a visual perspective: Each vdev gets a block of data and ZFS writes that data to all of (or in this case, both of) the disks in the mirror. As long as we have at least one functional disk in each vdev, we can retrieve our rainbow. As before, the chain link icons denote the disks are part of a single vdev. This configuration emphasizes performance over raw capacity but doesn’t totally disregard fault tolerance as our striped pool did. It’s a very popular configuration for systems that need a lot of fast I/O. Let’s look at one more example configuration using four 3-way mirrors. We’ll skip the individual vdev performance calculation and go straight to the full pool: 4x 3-way mirror: Read IOPS: 3000 Write IOPS: 1000 Streaming read speed: 3000 MB/s Streaming write speed: 400 MB/s Storage space efficiency: 33% (24 TB) Fault tolerance: 2 per vdev, 8 total While we have sacrificed some write performance and capacity, the pool is now extremely fault tolerant. This configuration is probably not practical for most applications and it would make more sense to use lower fault tolerance and set up an offsite backup system. Striped and mirrored vdevs are fantastic for access speed performance, but they either leave you with no redundancy whatsoever or impose at least a 50% penalty on the total usable space of your pool. In the next post, we will cover RAIDZ, which lets you keep data redundancy without sacrificing as much storage space efficiency. We’ll also look at some example workload scenarios and decide which layout would be the best fit for each. ###2FA with ssh on OpenBSD Five years ago I wrote about using a yubikey on OpenBSD. The only problem with doing this is that there’s no validation server available on OpenBSD, so you need to use a different OTP slot for each machine. (You don’t want to risk a replay attack if someone succeeds in capturing an OTP on one machine, right?) Yubikey has two OTP slots per device, so you would need a yubikey for every two machines with which you’d like to use it. You could use a bastion—and use only one yubikey—but I don’t like the SPOF aspect of a bastion. YMMV. After I played with TOTP, I wanted to use them as a 2FA for ssh. At the time of writing, we can’t do that using only the tools in base. This article focuses on OpenBSD; if you use another operating system, here are two handy links. SEED CONFIGURATION The first thing we need to do is to install the software which will be used to verify the OTPs we submit. # pkgadd loginoath We need to create a secret - aka, the seed - that will be used to calculate the Time-based One-Time Passwords. We should make sure no one can read or change it. $ openssl rand -hex 20 > ~/.totp-key $ chmod 400 ~/.totp-key Now we have a hexadecimal key, but apps usually want a base32 secret. I initially wrote a small script to do the conversion. While writing this article, I took the opportunity to improve it. When I initially wrote this utility for my use, python-qrcode hadn’t yet been imported to the OpenBSD ports/packages system. It’s easy to install now, so let’s use it. Here’s the improved version. It will ask for the hex key and output the secret as a base32-encoded string, both with and without spacing so you can copy-paste it into your password manager or easily retype it. It will then ask for the information needed to generate a QR code. Adding our new OTP secret to any mobile app using the QR code will be super easy! SYSTEM CONFIGURATION We can now move to the configuration of the system to put our new TOTP to use. As you might guess, it’s going to be quite close to what we did with the yubikey. We need to tweak login.conf. Be careful and keep a root shell open at all times. The few times I broke my OpenBSD were because I messed with login.conf without showing enough care. SSHD CONFIGURATION Again, keeping a root shell around decreases the risk of losing access to the system and being locked outside. A good standard is to use PasswordAuthentication no and to use public key only. Except… have a guess what the P stands for in TOTP. Yes, congrats, you guessed it! We need to switch to PasswordAuthentication yes. However, if we made this change alone, sshd would then accept a public key OR a password (which are TOTP because of our login.conf). 2FA uses both at the same time. To inform sshd we intend to use both, we need to set AuthenticationMethods publickey,password. This way, the user trying to login will first need to perform the traditional publickey authentication. Once that’s done, ssh will prompt for a password and the user will need to submit a valid TOTP for the system. We could do this the other way around, but I think bots could try passwords, wasting resources. Evaluated in this order, failing to provide a public key leads to sshd immediately declining your attempt. IMPROVING SECURITY WITHOUT IMPACTING UX My phone has a long enough password that most of the time, I fail to type it correctly on the first try. Of course, if I had to unlock my phone, launch my TOTP app and use my keyboard to enter what I see on my phone’s screen, I would quickly disable 2FA. To find a balance, I have whitelisted certain IP addresses and users. If I connect from a particular IP address or as a specific user, I don’t want to go through 2FA. For some users, I might not even enable 2FA. To sum up, we covered how to create a seed, how to perform a hexadecimal to base32 conversion and how to create a QR code for mobile applications. We configured the login system with login.conf so that ssh authentication uses the TOTP login system, and we told sshd to ask for both the public key and the Time-based One-Time Password. Now you should be all set to use two-factor ssh authentication on OpenBSD! ##News Roundup How ZFS maintains file type information in directories As an aside in yesterday’s history of file type information being available in Unix directories, I mentioned that it was possible for a filesystem to support this even though its Unix didn’t. By supporting it, I mean that the filesystem maintains this information in its on disk format for directories, even though the rest of the kernel will never ask for it. This is what ZFS does. The easiest way to see that ZFS does this is to use zdb to dump a directory. I’m going to do this on an OmniOS machine, to make it more convincing, and it turns out that this has some interesting results. Since this is OmniOS, we don’t have the convenience of just naming a directory in zdb, so let’s find the root directory of a filesystem, starting from dnode 1 (as seen before). # zdb -dddd fs3-corestaff-01/h/281 1 Dataset [....] [...] microzap: 512 bytes, 4 entries [...] ROOT = 3 # zdb -dddd fs3-corestaff-01/h/281 3 Object lvl iblk dblk dsize lsize %full type 3 1 16K 1K 8K 1K 100.00 ZFS directory [...] microzap: 1024 bytes, 8 entries RESTORED = 4396504 (type: Directory) ckstst = 12017 (type: not specified) ckstst3 = 25069 (type: Directory) .demo-file = 5832188 (type: Regular File) .peergroup = 12590 (type: not specified) cks = 5 (type: not specified) cksimap1 = 5247832 (type: Directory) .diskuse = 12016 (type: not specified) ckstst2 = 12535 (type: not specified) This is actually an old filesystem (it dates from Solaris 10 and has been transferred around with ‘zfs send | zfs recv’ since then), but various home directories for real and test users have been created in it over time (you can probably guess which one is the oldest one). Sufficiently old directories and files have no file type information, but more recent ones have this information, including .demo-file, which I made just now so this would have an entry that was a regular file with type information. Once I dug into it, this turned out to be a change introduced (or activated) in ZFS filesystem version 2, which is described in ‘zfs upgrade -v’ as ‘enhanced directory entries’. As an actual change in (Open)Solaris, it dates from mid 2007, although I’m not sure what Solaris release it made it into. The upshot is that if you made your ZFS filesystem any time in the last decade, you’ll have this file type information in your directories. How ZFS stores this file type information is interesting and clever, especially when it comes to backwards compatibility. I’ll start by quoting the comment from zfs_znode.h: /* * The directory entry has the type (currently unused on * Solaris) in the top 4 bits, and the object number in * the low 48 bits. The "middle" 12 bits are unused. */ In yesterday’s entry I said that Unix directory entries need to store at least the filename and the inode number of the file. What ZFS is doing here is reusing the 64 bit field used for the ‘inode’ (the ZFS dnode number) to also store the file type, because it knows that object numbers have only a limited range. This also makes old directory entries compatible, by making type 0 (all 4 bits 0) mean ‘not specified’. Since old directory entries only stored the object number and the object number is 48 bits or less, the higher bits are guaranteed to be all zero. The reason this needed a new ZFS filesystem version is now clear. If you tried to read directory entries with file type information on a version of ZFS that didn’t know about them, the old version would likely see crazy (and non-existent) object numbers and nothing would work. In order to even read a ‘file type in directory entries’ filesystem, you need to know to only look at the low 48 bits of the object number field in directory entries. ###Everything old is new again Just because KDE4-era software has been deprecated by the KDE-FreeBSD team in the official ports-repository, doesn’t mean we don’t care for it while we still need to. KDE4 was released on January 11th, 2008 — I still have the T-shirt — which was a very different C++ world than what we now live in. Much of the code pre-dates the availability of C11 — certainly the availability of compilers with C11 support. The language has changed a great deal in those ten years since the original release. The platforms we run KDE code on have, too — FreeBSD 12 is a long way from the FreeBSD 6 or 7 that were current at release (although at the time, I was more into OpenSolaris). In particular, since then the FreeBSD world has switched over to Clang, and FreeBSD current is experimenting with Clang 7. So we’re seeing KDE4-era code being built, and running, on FreeBSD 12 with Clang 7. That’s a platform with a very different idea of what constitutes correct code, than what the code was originally written for. (Not quite as big a difference as Helio’s KDE1 efforts, though) So, while we’re counting down to removing KDE4 from the FreeBSD ports tree, we’re also going through and fixing it to work with Clang 7, which defaults to a newer C++ standard and which is quite picky about some things. Some time in the distant past, when pointers were integers and NULL was zero, there was some confusion about booleans. So there’s lots of code that does list.contains(element) > 0 … this must have been a trick before booleans were a supported type in all our compilers. In any case it breaks with Clang 7, since contains() returns a QBool which converts to a nullptr (when false) which isn’t comparable to the integer 0. Suffice to say I’ve spent more time reading KDE4-era code this month, than in the past two years. However, work is proceeding apace, so if you really really want to, you can still get your old-school kicks on a new platform. Because we care about packaging things right, even when we want to get rid of it. ###OpenBSD netcat demystified Owing to its versatile functionalities, netcat earns the reputation as “TCP/IP Swiss army knife”. For example, you can create a simple chat app using netcat: (1) Open a terminal and input following command: # nc -l 3003 This means a netcat process will listen on 3003 port in this machine (the IP address of current machine is 192.168.35.176). (2) Connect aforemontioned netcat process in another machine, and send a greeting: # nc 192.168.35.176 3003 hello Then in the first machine’s terminal, you will see the “hello” text: # nc -l 3003 hello A primitive chatroom is built successfully. Very cool! Isn’t it? I think many people can’t wait to explore more features of netcatnow. If you are among them, congratulations! This tutorial may be the correct place for you. In the following parts, I will delve into OpenBSD’s netcatcode to give a detailed anatomy of it. The reason of picking OpenBSD’s netcat rather than others’ is because its code repository is small (~2000 lines of code) and neat. Furthermore, I also hope this little book can assist you learn more socket programming knowledge not just grasping usage of netcat. We’re all set. Let’s go! ##Beastie Bits What’s in store for NetBSD 9.0 NetBSD machines at Open Source Conference 2018 Hiroshima nmctl adapted with limited privileges: nmctl-0.6.0 Submit Your Work: Check out SCALE 17x and FOSDEM ’19 CFPs OpenBSD 6.4 site is up! (with a partial list of new features) Using Alpine to Read Your Email on OpenBSD ##Feedback/Questions Morgan - Send/Receive to Manage Fragmentation? Ryan - ZFS and mmap Marcus - Linux Compat Ben - Multiple Pools Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Episode 267: Absolute FreeBSD | BSD Now 267
We have a long interview with fiction and non-fiction author Michael W. Lucas for you this week as well as questions from the audience. ##Headlines Interview - Michael W. Lucas - mwlucas@michaelwlucas.com / @mwlauthor BR: [Welcome Back] AJ: What have you been doing since last we talked to you [ed, ssh, and af3e] BR: Tell us more about AF3e AJ: How did the first Absolute FreeBSD come about? BR: Do you have anything special planned for MeetBSD? AJ: What are you working on now? [FM:Jails, Git sync Murder] BR: What are your plans for next year? AJ: How has SEMIBug been going? Auction at https://mwl.io Patreon Link: ##Feedback/Questions Paul - Recent bhyve related videos (daemon) Michael - freebsd-update question Sigflup - pkg file search Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Episode 266: File Type History | BSD Now 266
Running OpenBSD/NetBSD on FreeBSD using grub2-bhyve, vermaden’s FreeBSD story, thoughts on OpenBSD on the desktop, history of file type info in Unix dirs, Multiboot a Pinebook KDE neon image, and more. ##Headlines OpenBSD/NetBSD on FreeBSD using grub2-bhyve When I was writing a blog post about the process title, I needed a couple of virtual machines with OpenBSD, NetBSD, and Ubuntu. Before that day I mainly used FreeBSD and Windows with bhyve. I spent some time trying to set up an OpenBSD using bhyve and UEFI as described here. I had numerous problems trying to use it, and this was the day I discovered the grub2-bhyve tool, and I love it! The grub2-bhyve allows you to load a kernel using GRUB bootloader. GRUB supports most of the operating systems with a standard configuration, so exactly the same method can be used to install NetBSD or Ubuntu. First, let’s install grub2-bhyve on our FreeBSD box: # pkg install grub2-bhyve To run grub2-bhyve we need to provide at least the name of the VM. In bhyve, if the memsize is not specified the default VM is created with 256MB of the memory. # grub-bhyve test GNU GRUB version 2.00 Minimal BASH-like line editing is supported. For the first word, TAB lists possible command completions. Anywhere else TAB lists possible device or file completions. grub> After running grub-bhyve command we will enter the GRUB loader. If we type the ls command, we will see all the available devices. In the case of the grub2-bhyve there is one additional device called “(host)” that is always available and allows the host filesystem to be accessed. We can list files under that device. grub> ls (host) grub> ls (host)/ libexec/ bin/ usr/ bhyve/ compat/ tank/ etc/ boot/ net/ entropy proc/ lib/ root/ sys/ mnt/ rescue/ tmp/ home/ sbin/ media/ jail/ COPYRIGHT var/ dev/ grub> To exit console simply type ‘reboot’. I would like to install my new operating system under a ZVOL ztank/bhyve/post. On another terminal, we create: # zfs create -V 10G ztank/bhyve/post If you don’t use ZFS for some crazy reason you can also create a raw blob using the truncate(1) command. # truncate -s 10G post.img I recommend installing an operating system from the disk image (installXX.fs for OpenBSD and NetBSD-X.X-amd64-install.img for NetBSD). Now we need to create a device map for a GRUB. cat > /tmp/post.map << EOF (hd0) /directory/to/disk/image (hd1) /dev/zvol/ztank/bhyve/post EOF The mapping files describe the names for files in the GRUB. In our case under hd0 we will have an installation image and in hd1 we will have our ZVOL/blob. You can also try to use an ISO image then instead of using hd0 device name use a cd0. When we will run the grub-bhyve command we will see two additional devices. # grub-bhyve -m /tmp/post.map post grub> ls (hd0) (hd0,msdos4) (hd0,msdos1) (hd0,openbsd9) (hd0,openbsd1) (hd1) (host) The hd0 (in this example OpenBSD image) contains multiple partitions. We can check what is on it. grub> ls (hd0,msdos4)/ boot bsd 6.4/ etc/ And this is the partition that contains a kernel. Now we can set a root device, load an OpenBSD kernel and boot: grub> set root=(hd0,msdos4) grub> kopenbsd -h com0 -r sd0a /bsd grub> boot After that, we can run bhyve virtual machine. In my case it is: # bhyve -c 1 -w -u -H \ -s 0,amd_hostbridge \ -s 3,ahci-hd,/directory/to/disk/image \ -s 4,ahci-hd,/dev/zvol/ztank/bhyve/post \ -s 31,lpc -l com1,stdio \ post Unfortunately explaining the whole bhyve(8) command line is beyond this article. After installing the operating system remove hd0 from the mapping file and the image from the bhyve(8) command. If you don’t want to type all those GRUB commands, you can simply redirect them to the standard input. cat << EOF | grub-bhyve -m /tmp/post.map -M 512 post set root=(hd0,4) kopenbsd -h com0 -r sd0a /bsd boot EOF ###My FreeBSD Story My first devices/computers/consoles (not at the same time) that I remember were Atari 2600 and Pegasus console which was hardware clone of the Nintendo NES. Back then I did not even knew that it was Atari 2600 as I referred to it as Video Computer System … and I did not even knew any english by then. It took me about two decades to get to know (by accident) that this Video Computer System was Atari 2600 Then I got AMIGA 600 computer (or should I say my parents bought it for me) which served both for playing computer games and also other activities for the first time. AMIGA is the computer that had the greatest influence on me, as it was the first time I studied the books about Amiga Workbench operating system and learned commands from Amiga Shell terminal. I loved the idea of Ram Disk icon/directory on the desktop that allowed me to transparently put any things in system memory. I still miss that concept on today’s desktop systems … and I still remember how dismal I was when I watched Amiga Deathbed Vigil movie. At the end of 1998 I got my first PC that of course came with Windows and that computer served both as gaming machine and as well as typical tool. One time I dig into the internals with Windows Registry (which left me disgusted by its concepts and implementation) and its limited command line interface provided by CMD.EXE executable. I remember that the heart of this box was not the CPU or the motherboard but the graphics accelerator – the legendary 3Dfx Voodoo card. This company (3Dfx) – their attitude and philosophy – also left solid fingerprint on my way. Like AMIGA did. After ‘migration’ from AMIGA to PC it never again ‘felt right’. The games were cool but the Windows system was horrible. Time has passed and different Windows versions and hardware modifications took place. Windows XP felt really heavy at that time, not to mention Windows 2000 for example with even bigger hardware requirements. I also do not understand all the hate about Windows ME. It crashed with the same frequency as Windows 98 or later Windows 98 Second Edition but maybe my hardware was different ?? I do not have any ‘mine’ screenshots from that period as I lost all my 40 GB (huge then) drive of data when I moved/resized the partition with Partition Magic to get some more space from the less filled C: drive. That day I learned hard that “there are people who do backups and people who will do backups”. I never lost data again as I had multiple copies of my data, but the same as Netheril fall the lost data was was gone forever. I always followed various alternatives which led me to try Linux in 2003, after reading about various distributions philosophies I decided to run Slackware Linux with KDE 3. My buddy used Aurox Linux by then (one of the few Linux distributions from Poland) and encouraged me to do the same – especially in the context of fixing possible problems as he already knew it and also as he recently dumped Windows system. But Slackware sounded like a better idea so I took that path instead. At first I dual booted between Windows XP and Slackware Linux cause I had everything worked out on the Windows world while I often felt helpless in the Linux world, so I would reboot into Windows to play some games or find a solution for Linux problem if that was required. I remember how strange the concept of dual clipboards (PRIMARY and SECONDARY) was for me by then. I was amazed why ‘so much better’ system as Linux (at least marketed that way) needs a system tray program to literally manage the clipboard. On Windows it was obvious, you do [CTRL]+[C] to copy and [CTRL]+[V] to paste things, but on Linux there (no I know its X11 feature) there were two clipboards that were synchronized by this little system tray program from KDE 3. It was also unthinkable for me that I will ‘lost’ contents of last/recent [CTRL]+[C] operation if I close the application from which the copy was made. I settled down a little on Slackware but not for long. I really did not liked manual dependency management for packages for example. Also KDE 3 was really ugly and despite trying all possible options I was not able to tweak it into something nice looking. After half a year on Slackware I checked the Linux distributions again and decided to try Gentoo Linux. I definitely agree with the image below which visualizes Gentoo Linux experience, especially when You install it for he first time ?? Of course I went with the most hardcore version with self building Stage 1 (compiler and toolchain) which was horrible idea at that time because compilation on slow single core machine took forever … but after many hours I got Gentoo installed. I now have to decide which desktop environment to use. I have read a lot of good news about Fluxbox at that time so this is what I tried. It was very weird experience (to create everything in GUI from scratch) but very pleasant one. That recalled me the times of AMIGA … but Linux came in the way too much often. The more I dig into Gentoo Linux the more I read that lots of Gentoo features are based on FreeBSD solutions. Gentoo Portage is a clone of FreeBSD Ports. That ‘central’ /etc/rc.conf system configuration file concept was taken from FreeBSD as well. So I started to gather information about FreeBSD. The (then) FreeBSD website or FreeBSD Ports site (still) felt little outdated to say the least but that did not discouraged me. Somewhere in 2005 I installed FreeBSD 5.4 on my computer. The beginnings were hard, like the earlier step with Gentoo but similarly like Gentoo the FreeBSD project came with a lot of great documentation. While Gentoo documentation is concentrated within various Gentoo Wiki sites the FreeBSD project comes with ‘official’ documentation in the form of Handbook and FAQ. I remember my first questions at the now nonexistent BSDForums.org site – for example one of the first ones – how to scroll the terminal output in the plain console. I now know that I had to push Scroll Lock button but it was something totally new for me. Why FreeBSD and not OpenBSD or NetBSD? Probably because Gentoo based most their concepts on the FreeBSD solutions, so that led me to FreeBSD instead of the other BSD operating systems. Currently I still use FreeBSD but I keep an steady eye on the OpenBSD, HardenedBSD and DragonFly BSD solutions and improvements. As the migration path from Linux to FreeBSD is a lot easier – all configuration files from /home can be just copied – the migration was quite fast easy. I again had the Fluxbox configuration which I used on the Gentoo. Now – on FreeBSD – it started to fell even more like AMIGA times. Everything is/has been well thought and had its place and reason. The documentation was good and the FreeBSD Community was second to none. After 15 years of using various Windows, UNIX (macOS/AIX/HP-UX/Solaris/OpenSolaris/Illumos/FreeBSD/OpenBSD/NetBSD) and UNIX-like (Linux) systems I always come to conclusion that FreeBSD is the system that sucks least. And sucks least with each release and one day I will write why FreeBSD is such great operating system … if I already haven’t ##News Roundup OpenBSD on the Desktop: some thoughts I’ve been using OpenBSD on my ThinkPad X230 for some weeks now, and the experience has been peculiar in some ways. The OS itself in my opinion is not ready for widespread desktop usage, and the development team is not trying to push it in the throat of anybody who wants a Windows or macOS alternative. You need to understand a little bit of how *NIX systems work, because you’ll use CLI more than UI. That’s not necessarily bad, and I’m sure I learned a trick or two that could translate easily to Linux or macOS. Their development process is purely based on developers that love to contribute and hack around, just because it’s fun. Even the mailing list is a cool place to hang on! Code correctness and security are a must, nothing gets committed if it doesn’t get reviewed thoroughly first - nowadays the first two properties should be enforced in every major operating system. I like the idea of a platform that continually evolves. pledge(2) and unveil(2) are the proof that with a little effort, you can secure existing software better than ever. I like the “sensible defaults” approach, having an OS ready to be used - UI included if you selected it during the setup process - is great. Just install a browser and you’re ready to go. Manual pages on OpenBSD are real manuals, not an extension of the “–help” command found in most CLI softwares. They help you understand inner workings of the operating system, no internet connection needed. There are some trade-offs, too. Performance is not first-class, mostly because of all the security mitigations and checks done at runtime. I write Go code in neovim, and sometimes you can feel a slight slowdown when you’re compiling and editing multiple files at the same time, but usually I can’t notice any meaningful difference. Browsers are a different matter though, you can definitely feel something differs from the experience you can have on mainstream operating systems. But again, trade-offs. To use OpenBSD on the desktop you must be ready to sacrifice some of the goodies of mainstream OSes, but if you’re searching for a zen place to do your computing stuff, it’s the best you can get right now. ###The history of file type information being available in Unix directories The two things that Unix directory entries absolutely have to have are the name of the directory entry and its ‘inode’, by which we generically mean some stable kernel identifier for the file that will persist if it gets renamed, linked to other directories, and so on. Unsurprisingly, directory entries have had these since the days when you read the raw bytes of directories with read(), and for a long time that was all they had; if you wanted more than the name and the inode number, you had to stat() the file, not just read the directory. Then, well, I’ll quote myself from an old entry on a find optimization: […], Unix filesystem developers realized that it was very common for programs reading directories to need to know a bit more about directory entries than just their names, especially their file types (find is the obvious case, but also consider things like ‘ls -F’). Given that the type of an active inode never changes, it’s possible to embed this information straight in the directory entry and then return this to user level, and that’s what developers did; on some systems, readdir(3) will now return directory entries with an additional dtype field that has the directory entry’s type. On Twitter, I recently grumbled about Illumos not having this dtype field. The ensuing conversation wound up with me curious about exactly where dtype came from and how far back it went. The answer turns out to be a bit surprising due to there being two sides of dtype. On the kernel side, dtype appears to have shown up in 4.4 BSD. The 4.4 BSD /usr/src/sys/dirent.h has a struct dirent that has a dtype field, but the field isn’t documented in either the comments in the file or in the getdirentries(2) manpage; both of those admit only to the traditional BSD dirent fields. This 4.4 BSD dtype was carried through to things that inherited from 4.4 BSD (Lite), specifically FreeBSD, but it continued to be undocumented for at least a while. (In FreeBSD, the most convenient history I can find is here, and the dtype field is present in sys/dirent.h as far back as FreeBSD 2.0, which seems to be as far as the repo goes for releases.) Documentation for dtype appeared in the getdirentries(2) manpage in FreeBSD 2.2.0, where the manpage itself claims to have been updated on May 3rd 1995 (cf). In FreeBSD, this appears to have been part of merging 4.4 BSD ‘Lite2’, which seems to have been done in 1997. I stumbled over a repo of UCB BSD commit history, and in it the documentation appears in this May 3rd 1995 change, which at least has the same date. It appears that FreeBSD 2.2.0 was released some time in 1997, which is when this would have appeared in an official release. In Linux, it seems that a dirent structure with a dtype member appeared only just before 2.4.0, which was released at the start of 2001. Linux took this long because the dtype field only appeared in the 64-bit ‘large file support’ version of the dirent structure, and so was only return by the new 64-bit getdents64() system call. This would have been a few years after FreeBSD officially documented dtype, and probably many years after it was actually available if you peeked at the structure definition. As far as I can tell, dtype is present on Linux, FreeBSD, OpenBSD, NetBSD, Dragonfly BSD, and Darwin (aka MacOS or OS X). It’s not present on Solaris and thus Illumos. As far as other commercial Unixes go, you’re on your own; all the links to manpages for things like AIX from my old entry on the remaining Unixes appear to have rotted away. Sidebar: The filesystem also matters on modern Unixes Even if your Unix supports dtype in directory entries, it doesn’t mean that it’s supported by the filesystem of any specific directory. As far as I know, every Unix with dtype support has support for it in their normal local filesystems, but it’s not guaranteed to be in all filesystems, especially non-Unix ones like FAT32. Your code should always be prepared to deal with a file type of DTUNKNOWN. It’s also possible to have things the other way around, where you have a filesystem with support for file type information in directories that’s on a Unix that doesn’t support it. There are a number of plausible reasons for this to happen, but they’re either obvious or beyond the scope of this entry. ###Multiboot Pinebook KDE neon Recently a KDE neon image for the Pinebook was announced. There is a new image, with a handful of fixes, which the KDE Plasma team has been working on over the past week and a half. Here’s a picture of my Pinebook running KDE neon — watching Panic! At the Disco’s High Hopes — sitting in front of my monitor that’s hooked up to one of my openSUSE systems. There are still some errata, and watching video sucks up battery, but for hacking on documentation from my hammock in the garden, or doing IRC meetings it’s a really nice machine. But one of the neat things about running KDE neon off of an SD card on the Pinebook is that it’s portable — that SD card can move around. So let’s talk about multiboot in the sense of “booting the same OS storage medium in different hardware units” rather than “booting different OS from a medium in a single hardware unit”. On these little ARM boards, u-boot does all the heavy lifting early in the boot process. So to re-use the KDE neon Pinebook image on another ARM board, the u-boot blocks need to be replaced. I have the u-boot from a Pine64 image (I forget what) lying around, 1015 blocks of 1024 bytes, which I can dd over the u-boot blocks on the SD card, dd bs=1k conv=notrunc,sync if=uboot.img of=/dev/da0 seek=8, and then the same SD card, with the filesystem and data from the Pinebook, will boot on the Pine64 board. Of course, to move the SD card back again, I need to restore the Pinebook u-boot blocks. Here’s a picture of my Pineboard (the base is a piece of the garden fence, it’s Douglas pine, with 4mm threaded rods acting as the corner posts for my Pine64 mini-rack), with power and network and a serial console attached, along with the serial console output of the same. The nice thing here is that the same software stack runs on the Pine64 but then has a wired network — which in turn means that if I switch on the other boards in that mini-rack, I’ve got a distcc-capable cluster for fast development, and vast NFS storage (served from ZFS on my FreeBSD machines) for source. I can develop in a high(er) powered environment, and then swap the card around into the Pinebook for testing-on-the-go. So to sum up: you can multiboot the KDE neon Pinebook image on other Pine64 hardware (i.e. the Pine64 board). To do so, you need to swap around u-boot blocks. The blocks can be picked out of an image built for each board, and then a particular image (e.g. the latest KDE neon Pinebook) can be run on either board. ##Beastie Bits Unexpected benefit with Ryzen – reducing power for build server Happy #CIDRDay! Absolute FreeBSD 3e ship date MWL FreeBSD talk @ October 9th 2018 - MUG Meeting MeetBSD Oct 19-20 October’s London *BSD meetup - 9th Oct 2018 NRW BUG Meeting at Trivago Oct. 9 Lars Wittebrood blogs about his visit to EuroBSDCon 2018 EuroBSDcon 2018 OpenBSD slides available EuroBSDCon conference site has most slides as well ##Feedback/Questions Brad - Unmounted ZFS sends Niclas - Report from a Meetup Ghislain - Bhyve not used? Shane - zpool history and snapshots Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Episode 265: Software Disenchantment | BSD Now 265
We report from our experiences at EuroBSDcon, disenchant software, LLVM 7.0.0 has been released, Thinkpad BIOS update options, HardenedBSD Foundation announced, and ZFS send vs. rsync. ##Headlines ###[FreeBSD DevSummit & EuroBSDcon 2018 in Romania] Your hosts are back from EuroBSDcon 2018 held in Bucharest, Romania this year. The first two days of the conference are used for tutorials and devsummits (FreeBSD and NetBSD), while the last two are for talks. Although Benedict organized the devsummit in large parts, he did not attend it this year. He held his Ansible tutorial in the morning of the first day, followed by Niclas Zeising’s new ports and poudriere tutorial (which had a record attendance). It was intended for beginners that had never used poudriere before and those who wanted to create their first port. The tutorial was well received and Niclas already has ideas for extending it for future conferences. On the second day, Benedict took Kirk McKusick’s “An Introduction to the FreeBSD Open-Source Operating System” tutorial, held as a one full day class this year. Although it was reduced in content, it went into enough depth of many areas of the kernel and operating system to spark many questions from attendees. Clearly, this is a good start into kernel programming as Kirk provides enough material and backstories to understand why certain things are implemented as they are. Olivier Robert took https://www.talegraph.com/tales/l2o9ltrvsE (pictures from the devsummit) and created a nice gallery out of it. Devsummit evenings saw dinners at two restaurants that allowed developers to spend some time talking over food and drinks. The conference opened on the next day with the opening session held by Mihai Carabas. He introduced the first keynote speaker, a colleague of his who presented “Lightweight virtualization with LightVM and Unikraft”. Benedict helped out at the FreeBSD Foundation sponsor table and talked to people. He saw the following talks in between: Selfhosting as an alternative to the public cloud (by Albert Dengg) Using Boot Environments at Scale (by Allan Jude) Livepatching FreeBSD kernel (by Maciej Grochowski) FreeBSD: What to (Not) Monitor (by Andrew Fengler) FreeBSD Graphics (by Niclas Zeising) Allan spent a lot of time talking to people and helping track down issues they were having, in addition to attending many talks: Hacking together a FreeBSD presentation streaming box – For as little as possible (by Tom Jones) Introduction of FreeBSD in new environments (by Baptiste Daroussin) Keynote: Some computing and networking historical perspectives (by Ron Broersma) Livepatching FreeBSD kernel (by Maciej Grochowski) FreeBSD: What to (Not) Monitor (by Andrew Fengler) Being a BSD user (by Roller Angel) From “Hello World” to the VFS Layer: building a beadm for DragonFly BSD (by Michael Voight) We also met the winner of our Power Bagel raffle from Episode 2^8. He received the item in the meantime and had it with him at the conference, providing a power outlet to charge other people’s devices. During the closing session, GroffTheBSDGoat was handed over to Deb Goodkin, who will bring the little guy to the Grace Hopper Celebration of Women in Computing conference and then to MeetBSD later this year. It was also revealed that next year’s EuroBSDcon will be held in Lillehammer, Norway. Thanks to all the speakers, helpers, sponsors, organizers, and attendees for making it a successful conferences. There were no talks recorded this year, but the slides will be uploaded to the EuroBSDcon website in a couple of weeks. The OpenBSD talks are already available, so check them out. ###Software disenchantment I’ve been programming for 15 years now. Recently our industry’s lack of care for efficiency, simplicity, and excellence started really getting to me, to the point of me getting depressed by my own career and the IT in general. Modern cars work, let’s say for the sake of argument, at 98% of what’s physically possible with the current engine design. Modern buildings use just enough material to fulfill their function and stay safe under the given conditions. All planes converged to the optimal size/form/load and basically look the same. Only in software, it’s fine if a program runs at 1% or even 0.01% of the possible performance. Everybody just seems to be ok with it. People are often even proud about how much inefficient it is, as in “why should we worry, computers are fast enough”: @tveastman: I have a Python program I run every day, it takes 1.5 seconds. I spent six hours re-writing it in rust, now it takes 0.06 seconds. That efficiency improvement means I’ll make my time back in 41 years, 24 days :-) You’ve probably heard this mantra: “programmer time is more expensive than computer time”. What it means basically is that we’re wasting computers at an unprecedented scale. Would you buy a car if it eats 100 liters per 100 kilometers? How about 1000 liters? With computers, we do that all the time. Everything is unbearably slow Look around: our portable computers are thousands of times more powerful than the ones that brought man to the moon. Yet every other webpage struggles to maintain a smooth 60fps scroll on the latest top-of-the-line MacBook Pro. I can comfortably play games, watch 4K videos but not scroll web pages? How is it ok? Google Inbox, a web app written by Google, running in Chrome browser also by Google, takes 13 seconds to open moderately-sized emails: It also animates empty white boxes instead of showing their content because it’s the only way anything can be animated on a webpage with decent performance. No, decent doesn’t mean 60fps, it’s rather “as fast as this web page could possibly go”. I’m dying to see web community answer when 120Hz displays become mainstream. Shit barely hits 60Hz already. Windows 10 takes 30 minutes to update. What could it possibly be doing for that long? That much time is enough to fully format my SSD drive, download a fresh build and install it like 5 times in a row. Pavel Fatin: Typing in editor is a relatively simple process, so even 286 PCs were able to provide a rather fluid typing experience. Modern text editors have higher latency than 42-year-old Emacs. Text editors! What can be simpler? On each keystroke, all you have to do is update tiny rectangular region and modern text editors can’t do that in 16ms. It’s a lot of time. A LOT. A 3D game can fill the whole screen with hundreds of thousands (!!!) of polygons in the same 16ms and also process input, recalculate the world and dynamically load/unload resources. How come? As a general trend, we’re not getting faster software with more features. We’re getting faster hardware that runs slower software with the same features. Everything works way below the possible speed. Ever wonder why your phone needs 30 to 60 seconds to boot? Why can’t it boot, say, in one second? There are no physical limitations to that. I would love to see that. I would love to see limits reached and explored, utilizing every last bit of performance we can get for something meaningful in a meaningful way. Everything is HUUUUGE And then there’s bloat. Web apps could open up to 10× faster if you just simply block all ads. Google begs everyone to stop shooting themselves in their feet with AMP initiative—a technology solution to a problem that doesn’t need any technology, just a little bit of common sense. If you remove bloat, the web becomes crazy fast. How smart do you have to be to understand that? Android system with no apps takes almost 6 Gb. Just think for a second how obscenely HUGE that number is. What’s in there, HD movies? I guess it’s basically code: kernel, drivers. Some string and resources too, sure, but those can’t be big. So, how many drivers do you need for a phone? Windows 95 was 30Mb. Today we have web pages heavier than that! Windows 10 is 4Gb, which is 133 times as big. But is it 133 times as superior? I mean, functionally they are basically the same. Yes, we have Cortana, but I doubt it takes 3970 Mb. But whatever Windows 10 is, is Android really 150% of that? Google keyboard app routinely eats 150 Mb. Is an app that draws 30 keys on a screen really five times more complex than the whole Windows 95? Google app, which is basically just a package for Google Web Search, is 350 Mb! Google Play Services, which I do not use (I don’t buy books, music or videos there)—300 Mb that just sit there and which I’m unable to delete. All that leaves me around 1 Gb for my photos after I install all the essential (social, chats, maps, taxi, banks etc) apps. And that’s with no games and no music at all! Remember times when an OS, apps and all your data fit on a floppy? Your desktop todo app is probably written in Electron and thus has userland driver for Xbox 360 controller in it, can render 3d graphics and play audio and take photos with your web camera. A simple text chat is notorious for its load speed and memory consumption. Yes, you really have to count Slack in as a resource-heavy application. I mean, chatroom and barebones text editor, those are supposed to be two of the less demanding apps in the whole world. Welcome to 2018. At least it works, you might say. Well, bigger doesn’t imply better. Bigger means someone has lost control. Bigger means we don’t know what’s going on. Bigger means complexity tax, performance tax, reliability tax. This is not the norm and should not become the norm. Overweight apps should mean a red flag. They should mean run away scared. Better world manifesto I want to see progress. I want change. I want state-of-the-art in software engineering to improve, not just stand still. I don’t want to reinvent the same stuff over and over, less performant and more bloated each time. I want something to believe in, a worthy end goal, a future better than what we have today, and I want a community of engineers who share that vision. What we have today is not progress. We barely meet business goals with poor tools applied over the top. We’re stuck in local optima and nobody wants to move out. It’s not even a good place, it’s bloated and inefficient. We just somehow got used to it. So I want to call it out: where we are today is bullshit. As engineers, we can, and should, and will do better. We can have better tools, we can build better apps, faster, more predictable, more reliable, using fewer resources (orders of magnitude fewer!). We need to understand deeply what are we doing and why. We need to deliver: reliably, predictably, with topmost quality. We can—and should–take pride in our work. Not just “given what we had…”—no buts! I hope I’m not alone at this. I hope there are people out there who want to do the same. I’d appreciate if we at least start talking about how absurdly bad our current situation in the software industry is. And then we maybe figure out how to get out. ##News Roundup [llvm-announce] LLVM 7.0.0 Release I am pleased to announce that LLVM 7 is now available. Get it here: https://llvm.org/releases/download.html#7.0.0 The release contains the work on trunk up to SVN revision 338536 plus work on the release branch. It is the result of the community's work over the past six months, including: function multiversioning in Clang with the 'target' attribute for ELF-based x86/x86_64 targets, improved PCH support in clang-cl, preliminary DWARF v5 support, basic support for OpenMP 4.5 offloading to NVPTX, OpenCL C++ support, MSan, X-Ray and libFuzzer support for FreeBSD, early UBSan, X-Ray and libFuzzer support for OpenBSD, UBSan checks for implicit conversions, many long-tail compatibility issues fixed in lld which is now production ready for ELF, COFF and MinGW, new tools llvm-exegesis, llvm-mca and diagtool. And as usual, many optimizations, improved diagnostics, and bug fixes. For more details, see the release notes: https://llvm.org/releases/7.0.0/docs/ReleaseNotes.html https://llvm.org/releases/7.0.0/tools/clang/docs/ReleaseNotes.html https://llvm.org/releases/7.0.0/tools/clang/tools/extra/docs/ReleaseNotes.html https://llvm.org/releases/7.0.0/tools/lld/docs/ReleaseNotes.html Thanks to everyone who helped with filing, fixing, and code reviewing for the release-blocking bugs! Special thanks to the release testers and packagers: Bero Rosenkränzer, Brian Cain, Dimitry Andric, Jonas Hahnfeld, Lei Huang Michał Górny, Sylvestre Ledru, Takumi Nakamura, and Vedant Kumar. For questions or comments about the release, please contact the community on the mailing lists. Onwards to LLVM 8! Cheers, Hans ###Update your Thinkpad’s bios with Linux or OpenBSD Get your new bios At first, go to the Lenovo website and download your new bios: Go to lenovo support Use the search bar to find your product (example for me, x270) Choose the right product (if necessary) and click search On the right side, click on Update Your System Click on BIOS/UEFI Choose *BIOS Update (Bootable CD) for Windows * Download For me the file is called like this : r0iuj25wd.iso Extract bios update Now you will need to install geteltorito. With OpenBSD: $ doas pkgadd geteltorito quirks-3.7 signed on 2018-09-09T13:15:19Z geteltorito-0.6: ok With Debian: $ sudo apt-get install genisoimage Now we will extract the bios update : $ geteltorito -o biosupdate.img r0iuj25wd.iso Booting catalog starts at sector: 20 Manufacturer of CD: NERO BURNING ROM VER 12 Image architecture: x86 Boot media type is: harddisk El Torito image starts at sector 27 and has 43008 sector(s) of 512 Bytes Image has been written to file "biosupdate.img". This will create a file called biosupdate.img. Put the image on an USB key CAREFULL : on my computer, my USB key is sda1 on Linux and sd1 on OpenBSD. Please check twice on your computer the name of your USB key. With OpenBSD : $ doas dd if=biosupdate.img of=/dev/rsd1c With Linux : $ sudo dd if=biosupdate.img of=/dev/sda Now all you need is to reboot, to boot on your USB key and follow the instructions. Enjoy 😉 ###Announcing The HardenedBSD Foundation In June of 2018, we announced our intent to become a not-for-profit, tax-exempt 501©(3) organization in the United States. It took a dedicated team months of work behind-the-scenes to make that happen. On 06 September 2018, HardenedBSD Foundation Corp was granted 501©(3) status, from which point all US-based persons making donations can deduct the donation from their taxes. We are grateful for those who contribute to HardenedBSD in whatever way they can. Thank you for making HardenedBSD possible. We look forward to a bright future, driven by a helpful and positive community. ###How you migrate ZFS filesystems matters If you want to move a ZFS filesystem around from one host to another, you have two general approaches; you can use ‘zfs send’ and ‘zfs receive’, or you can use a user level copying tool such as rsync (or ‘tar -cf | tar -xf’, or any number of similar options). Until recently, I had considered these two approaches to be more or less equivalent apart from their convenience and speed (which generally tilted in favour of ‘zfs send’). It turns out that this is not necessarily the case and there are situations where you will want one instead of the other. We have had two generations of ZFS fileservers so far, the Solaris ones and the OmniOS ones. When we moved from the first generation to the second generation, we migrated filesystems across using ‘zfs send’, including the filesystem with my home directory in it (we did this for various reasons). Recently I discovered that some old things in my filesystem didn’t have file type information in their directory entries. ZFS has been adding file type information to directories for a long time, but not quite as long as my home directory has been on ZFS. This illustrates an important difference between the ‘zfs send’ approach and the rsync approach, which is that zfs send doesn’t update or change at least some ZFS on-disk data structures, in the way that re-writing them from scratch from user level does. There are both positives and negatives to this, and a certain amount of rewriting does happen even in the ‘zfs send’ case (for example, all of the block pointers get changed, and ZFS will re-compress your data as applicable). I knew that in theory you had to copy things at the user level if you wanted to make sure that your ZFS filesystem and everything in it was fully up to date with the latest ZFS features. But I didn’t expect to hit a situation where it mattered in practice until, well, I did. Now I suspect that old files on our old filesystems may be partially missing a number of things, and I’m wondering how much of the various changes in ‘zfs upgrade -v’ apply even to old data. (I’d run into this sort of general thing before when I looked into ext3 to ext4 conversion on Linux.) With all that said, I doubt this will change our plans for migrating our ZFS filesystems in the future (to our third generation fileservers). ZFS sending and receiving is just too convenient, too fast and too reliable to give up. Rsync isn’t bad, but it’s not the same, and so we only use it when we have to (when we’re moving only some of the people in a filesystem instead of all of them, for example). PS: I was going to try to say something about what ‘zfs send’ did and didn’t update, but having looked briefly at the code I’ve concluded that I need to do more research before running my keyboard off. In the mean time, you can read the OpenZFS wiki page on ZFS send and receive, which has plenty of juicy technical details. PPS: Since eliminating all-zero blocks is a form of compression, you can turn zero-filled files into sparse files through a ZFS send/receive if the destination has compression enabled. As far as I know, genuine sparse files on the source will stay sparse through a ZFS send/receive even if they’re sent to a destination with compression off. ##Beastie Bits BSD Users Stockholm Meetup #4: Tuesday, November 13, 2018 at 18:00 BSD Poland User Group: Next Meeting: October 11, 2018, 18:15 - 21:15 at Warsaw University of Technology n2k18 Hackathon report: Ken Westerback (krw@) on disklabel(8) work, dhclient(8) progress Running MirageOS Unikernels on OpenBSD in vmm (Now Works) vmm(4) gets support for qcow2 MeetBSD and SecurityBsides Colin Percival reduced FreeBSD startup time from 10627ms (11.2) to 4738ms (12.0) FreeBSD 11.1 end-of-life KnoxBug: Monday, October 1, 2018 at 18:00: Real-world Performance Advantages of NVDIMM and NVMe: Case Study with OpenZFS ##Feedback/Questions Todd - 2 Nics, 1 bhyve and a jail cell Thomas - Deep Dive Morgan - Send/Receive to Manage Fragmentation? Dominik - hierarchical jails -> networking Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Episode 264: Optimized-out | BSD Now 264
FreeBSD and DragonflyBSD benchmarks on AMD’s Threadripper, NetBSD 7.2 has been released, optimized out DTrace kernel symbols, stuck UEFI bootloaders, why ed is not a good editor today, tell your BSD story, and more. ##Headlines FreeBSD & DragonFlyBSD Put Up A Strong Fight On AMD’s Threadripper 2990WX, Benchmarks Against Linux The past two weeks I have been delivering a great deal of AMD Threadripper 2990WX benchmarks on Linux as well as some against Windows and Windows Server. But recently I got around to trying out some of the BSD operating systems on this 32-core / 64-thread processor to see how they would run and to see whether they would have similar scaling issues or not like we’ve seen on the Windows side against Linux. In this article are FreeBSD and DragonFlyBSD benchmarks with the X399 + 2990WX compared to a few Linux distributions. The BSDs I focused my testing on were FreeBSD 11.2-STABLE and 12.0-CURRENT/ALPHA1 (the version in development) as well as iX System’s TrueOS that is tracking FreeBSD 12.0-CURRENT. Also included were DragonFlyBSD, with FreeBSD and DragonFlyBSD being tied as my favorite operating systems when it comes to the BSDs. When it came to FreeBSD 11.2-STABLE and 12.0-ALPHA1 on the Threadripper 2990WX, it worked out surprisingly well. I encountered no real issues during my two days of benchmarking on FreeBSD (and TrueOS). It was a great experience and FreeBSD was happy to exploit the 64 threads on the system. DragonFlyBSD was a bit of a different story… Last week when I started this BSD testing I tried DragonFly 5.2.2 as the latest stable release as well as a DragonFlyBSD 5.3 development snapshot from last week: both failed to boot in either BIOS or UEFI modes. But then a few days ago DragonFlyBSD lead developer Matthew Dillon bought himself a 2990WX platform. He made the necessary changes to get DragonFlyBSD 5.3 working and he ended up finding really great performance and potential out of the platform. So I tried the latest DragonFlyBSD 5.3 daily ISO on 22 August and indeed it now booted successfully and we were off to the races. Thus there are some DragonFlyBSD 5.3 benchmarks included in this article too. Just hours ago, Matthew Dillon landed some 2990WX topology and scheduler enhancements but that fell out of the scope of when DragonFly was installed on this system. But over the weekend or so I plan to re-test DragonFlyBSD 5.3 and see how those optimizations affect the overall 2990WX performance now on that BSD. DragonFlyBSD 5.4 stable should certainly be an interesting release on several fronts! With FreeBSD 11.2-STABLE and 12.0-ALPHA1 I ran benchmarks when using their stock compiler (LLVM Clang 6.0) as well as GCC 7.3 obtained via GCC 7.3. That was done to rule out compiler differences in benchmarking against the GCC-based Linux distributions. On DragonFlyBSD 5.3 it defaults to the GCC 5.4.1 but via pkg I also did a secondary run when upgraded to GCC 7.3. The hardware and BIOS/UEFI settings were maintained the same throughout the entire benchmarking process. The system was made up of the AMD Ryzen Threadripper 2990WX at stock speeds, the ASUS ROG ZENITH EXTREME motherboard, 4 x 8GB DDR4-3200MHz memory, Samsung 970 EVO 500GB NVMe SSD, and Radeon RX Vega 56 graphics card. All of these Linux vs. BSD benchmarks were carried out in a fully-automated and reproducible manner using the open-source Phoronix Test Suite benchmarking framework. While for the last of today’s BSD vs. Linux benchmarking on the Threadripper 2990WX, the Linux distributions came out slightly ahead of FreeBSD and DragonFlyBSD with GCC (another test having issues with Clang 6.0 on the BSDs). Overall, I was quite taken away by the BSD performance on the Threadripper 2990WX – particularly FreeBSD. In a surprising number of benchmarks, the BSDs were outperforming the tested Linux distributions though often by incredibly thin margins. Still, quite an accomplishment for these BSD operating systems and considering how much better Linux is already doing than Windows 10 / Windows Server on this 32-core / 64-thread processor. Then again, the BSDs like Linux have a long history of running on high core/thread-count systems, super computers, and other HPC environments. It will be interesting to see how much faster DragonFlyBSD can run given today’s commit to its kernel with scheduler and topology improvements for the 2990WX. Those additional DragonFlyBSD benchmarks will be published in the coming days once they are completed. ###NetBSD 7.2 released The NetBSD Project is pleased to announce NetBSD 7.2, the second feature update of the NetBSD 7 release branch. It represents a selected subset of fixes deemed important for security or stability reasons, as well as new features and enhancements. General Security Note The NetBSD 7.2 release is a maintenance release of the netbsd-7 branch, which had it's first major release, NetBSD 7.0 in September 2015. A lot of security features have been added to later NetBSD versions, and for new installations we highly recommend using our latest release, NetBSD 8.0 instead. Some highlights of the 7.2 release are: Support for USB 3.0. Enhancements to the Linux emulation subsystem. Fixes in binary compatibility for ancient NetBSD executables. iwm(4) driver for Intel Wireless 726x, 316x, 826x and 416x series added. Support for Raspberry Pi 3 added. Fix interrupt setup on Hyper-V VMs with Legacy Network Adapter. SVR4 and IBCS2 compatibility subsystems have been disabled by default (besides IBCS2 on VAX). These subsystems also do not auto-load their modules any more. Various USB stability enhancements. Numerous bug fixes and stability improvements. Complete source and binaries for NetBSD 7.2 are available for download at many sites around the world. A list of download sites providing FTP, AnonCVS, SUP, and other services may be found at https://www.NetBSD.org/mirrors/. We encourage users who wish to install via ISO or USB disk images to download via BitTorrent by using the torrent files supplied in the images area. A list of hashes for the NetBSD 7.2 distribution has been signed with the well-connected PGP key for the NetBSD Security Officer: https://cdn.NetBSD.org/pub/NetBSD/security/hashes/NetBSD-7.2_hashes.asc NetBSD is free. All of the code is under non-restrictive licenses, and may be used without paying royalties to anyone. Free support services are available via our mailing lists and website. Commercial support is available from a variety of sources. More extensive information on NetBSD is available from our website: ##News Roundup Including optimized-out kernel symbols in dtrace on FreeBSD Have you ever had dtrace(1) on FreeBSD fail to list a probe that should exist in the kernel? This is because Clang will optimize-out some functions. The result is ctfconvert(1) will not generate debugging symbols that dtrace(1) uses to identify probes. I have a quick solution to getting those probes visible to dtrace(1). In my case, I was trying to instrument on ieee80211_ioctl_get80211, whose sister function ieee80211_ioctl_set80211 has a dtrace(1) probe in the generic FreeBSD 11 and 12 kernels. Both functions are located in /usr/src/sys/net80211/ieee80211_ioctl.c. My first attempt was to add to /etc/make.conf as follows and recompile the kernel. CFLAGS+=-O0 and -fno-inline-functions This failed to produce the dtrace(1) probe. Several other attempts failed and I was getting inconsistent compilation results (Is it me or is ieee80211_ioctl.c compiled with different flags if NO_CLEAN=1 is set?). When I manually compiled the object file by copying the compilation line for the object file and adding -O0 -fno-inline-functions, nm(1) on both the object file and kernel demonstrated that the symbol was present. I installed the kernel, rebooted and it was listed as a dtrace probe. Great! But as I continued to debug my WiFi driver (oh yeah, I’m very slowly extending rtwn(4)), I found myself rebuilding the kernel several times and frequently rebooting. Why not do this across the entire kernel? After hacking around, my solution was to modify the build scripts. My solution was to edit /usr/src/sys/conf/kern.pre.mk and modify all optimization level 2 to optimization level 0. The following is my diff(1) on FreeBSD 12.0-CURRENT. A few thoughts: This seems like a hack rather than a long-term solution. Either the problem is with the hard-coded optimization flags, or the inability to overwrite them in all places in make.conf. Removing optimizations is only something I would do in a non-production kernel, so its as if I have to choose between optimizations for a production kernel or having dtrace probes. But dtrace explicitly markets itself as not impactful on production. Using the dtrace pony as your featured image on WordPress does not render properly and must be rotated and modified. Blame Bryan Cantrill. If you have a better solution, please let me know and I will update the article, but this works for me! ###FreeBSD: UEFI Bootloader stuck on BootCurrent/BootOrder/BootInfo on Asus Motherboards (and fix!) Starting with FreeBSD CURRENT from about a few weeks of posting date, but including FreeBSD 12 alpha releases (not related to DEC Alpha), I noticed one thing: When I boot FreeBSD from UEFI on a homebuilt desktop with a Asus H87M-E motherboard, and have Root on ZFS, the bootloader gets stuck on lines like BootCurrent, BootOrder, and BootInfo. This issue occurs when I try to boot directly to efi\boot\bootx64.efi. One person had a similar issue on a Asus H87I-PLUS motherboard. This issue may or may not exist on other Asus motherboards, desktops, or laptops. This may be specific to Asus motherboards for Intel’s Haswell, but may also exist on newer systems (e.g. Skylake) or older (e.g. Ivy Bridge) with Asus motherboards, as well as Asus desktops or laptops. There are two solutions to this problem: Use Legacy BIOS mode instead of UEFI mode Install a FreeBSD UEFI Boot entry Keep in mind that I am not going to talk about this issue and third-party UEFI boot managers such as rEFInd here. The first option is rather straightforward: you need to make sure your computer has “Secure Boot” disabled and “Legacy Boot” or “CSM” enabled. Then, you need to make sure FreeBSD is installed in BIOS mode. However, this solution is (in my opinion) suboptimal. Why? Because: You won’t be able to use hard drives bigger than 2TB You are limited to MBR Partitioning on Asus motherboards with UEFI as Asus motherboards refuse to boot GPT partitioned disks in BIOS mode Legacy BIOS mode may not exist on future computers or motherboards (although those systems may not have this issue, and this issue may get fixed by then) The second option, however, is less straightforward, but will let you keep UEFI. Many UEFI systems, including affected Asus motherboards described here, include a boot manager built into the UEFI. FreeBSD includes a tool called efibootmgr to manage this, similar to the similarly-named tool in Linux, but with a different syntax. ###Why ed(1) is not a good editor today I’ll start with my tweet: Heretical Unix opinion time: ed(1) may be the 'standard Unix editor', but it is not a particularly good editor outside of a limited environment that almost never applies today. There is a certain portion of Unixdom that really likes ed(1), the ‘standard Unix editor’. Having actually used ed for a not insignificant amount of time (although it was the friendlier ‘UofT ed’ variant), I have some reactions to what I feel is sometimes overzealous praise of it. One of these is what I tweeted. The fundamental limitation of ed is that it is what I call an indirect manipulation interface, in contrast to the explicit manipulation interfaces of screen editors like vi and graphical editors like sam (which are generally lumped together as ‘visual’ editors, so called because they actually show you the text you’re editing). When you edit text in ed, you have some problems that you don’t have in visual editors; you have to maintain in your head the context of what the text looks like (and where you are in it), you have to figure out how to address portions of that text in order to modify them, and finally you have to think about how your edit commands will change the context. Copious use of ed’s p command can help with the first problem, but nothing really deals with the other two. In order to use ed, you basically have to simulate parts of ed in your head. Ed is a great editor in situations where the editor explicitly presenting this context is a very expensive or outright impossible operation. Ed works great on real teletypes, for example, or over extremely slow links where you want to send and receive as little data as possible (and on real teletypes you have some amount of context in the form of an actual printout that you can look back at). Back in the old days of Unix, this described a fairly large number of situations; you had actual teletypes, you had slow dialup links (and later slow, high latency network links), and you had slow and heavily overloaded systems. However, that’s no longer the situation today (at least almost all of the time). Modern systems and links can easily support visual editors that continually show you the context of the text and generally let you more or less directly manipulate it (whether that is through cursoring around it or using a mouse). Such editors are easier and faster to use, and they leave you with more brainpower free to think about things like the program you’re writing (which is the important thing). If you can use a visual editor, ed is not a particularly good editor to use instead; you will probably spend a lot of effort (and some amount of time) on doing by hand something that the visual editor will do for you. If you are very practiced at ed, maybe this partly goes away, but I maintain that you are still working harder than you need to be. The people who say that ed is a quite powerful editor are correct; ed is quite capable (although sadly limited by only editing a single file). It’s just that it’s also a pain to use. (They’re also correct that ed is the foundation of many other things in Unix, including sed and vi. But that doesn’t mean that the best way to learn or understand those things is to learn and use ed.) This doesn’t make ed a useless, vestigial thing on modern Unix, though. There are uses for ed in non-interactive editing, for example. But on modern Unix, ed is a specialized tool, much like dc. It’s worth knowing that ed is there and roughly what it can do, but it’s probably not worth learning how to use it before you need it. And you’re unlikely to ever be in a situation where it’s the best choice for interactive editing (and if you are, something has generally gone wrong). (But if you enjoy exploring the obscure corners of Unix, sure, go for it. Learn dc too, because it’s interesting in its own way and, like ed, it’s one of those classical old Unix programs.) ##Beastie Bits Is there any interest in a #BSD user group in #Montreal? Tell your BSD story Finishing leftover tasks from Google Summer of Code Fuzzing the OpenBSD Kernel ARM - any Tier-1 *BSD options? ##Feedback/Questions Chris - byhve question Paulo - Topic suggestion Bostjan - How data gets to disk Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Episode 263: Encrypt That Pool | BSD Now 263
Mitigating Spectre/Meltdown on HP Proliant servers, omniOS installation setup, debugging a memory corruption issue on OpenBSD, CfT for OpenZFS native encryption, Asigra TrueNAS backup appliance shown at VMworld, NetBSD 6 EoL, and more. ##Headlines How to mitigate Spectre and Meltdown on an HP Proliant server with FreeBSD As recently announced in a previous article I wanted to write a couple of guides on how to mitigate Spectre and Meltdown vulnerabilities in GNU/Linux and UNIX environments. It is always a good and I hope a standard practice to have your systems patched and if they aren’t for whatever the reason (that legacy thing you’re carrying on for ages) you may take the necessary extra steps to protect your environment. I never planned to do any article on patching anything. Nowadays it’s a no brainer and operating systems have provided the necessary tools for this to be easy and as smooth as possible. So why this article? Spectre and Meltdown are both hardware vulnerabilities. Major ones. They are meaningful for several reasons among them the world wide impact since they affect Intel and AMD systems which are ubiquitous. And second because patching hardware is not as easy, for the manufacturer and for the users or administrators in charge of the systems. There is still no known exploit around left out in the open hitting servers or desktops anywhere. The question is not if it will ever happen. The question is when will it happen. And it may be sooner than later. This is why big companies, governments and people in charge of big deployments are patching or have already patched their systems. But have you done it to your system? I know you have a firewall. Have you thought about CVE-2018-3639? This particular one could make your browser being a vector to get into your system. So, no, there is no reason to skip this. Patching these set of vulnerabilities implies some more steps and concerns than updating the operating system. If you are a regular Windows user I find rare you to be here and many of the things you will read may be foreign to you. I am not planning to do a guide on Windows systems since I believe someone else has or will do it and will do it better than me since I am not a pro Windows user. However there is one basic and common thing for all OS’s when dealing with Spectre and Meltdown and that is a microcode update is necessary for the OS patches to effectively work. What is microcode? You can read the Wikipedia article but in short it is basically a layer of code that allows chip manufacturers to deal with modifications on the hardware they’ve produced and the operating systems that will manage that hardware. Since there’s been some issues (namely Spectre and Meltdown) Intel and AMD respectively have released a series of microcode updates to address those problems. First series did come with serious problems and some regressions, to the point GNU/Linux producers stopped releasing the microcode updates through their release channels for updates and placed the ball on Intel’s roof. Patching fast does always include risks, specially when dealing with hardware. OS vendors have resumed their microcode update releases so all seems to be fine now. In order to update the microcode we’re faced with two options. Download the most recent BIOS release from our vendor, provided it patches the Spectre and Meltdown vulnerabilities, or patch it from the OS. If your hardware vendor has decided not to provide support on your hardware you are forced to use the latter solution. Yes, you can still keep your hardware. They usually come accompanied with a “release notes” file where there are some explanatory notes on what is fixed, what is new, etc. To make the search easy for you a news site collected the vendors list and linked the right support pages for anyone to look. In some scenarios it would be desirable not to replace the whole BIOS but just update the microcode from the OS side. In my case I should update an HP Proliant ML110 G7 box and the download link for that would be this. Instead of using the full blown BIOS update path we’ll use the inner utilities to patch Spectre and Meltdown on FreeBSD. So let’s put our hands on it See the article for the technical breakdown ###A look beyond the BSD teacup: OmniOS installation Five years ago I wrote a post about taking a look beyond the Linux teacup. I was an Arch Linux user back then and since there were projects like ArchBSD (called PacBSD today) and Arch Hurd, I decided to take a look at and write about them. Things have changed. Today I’m a happy FreeBSD user, but it’s time again to take a look beyond the teacup of operating systems that I’m familiar with. Why Illumos / OmniOS? There are a couple of reasons. The Solaris derivatives are the other big community in the *nix family besides Linux and the BSDs and we hadn’t met so far. Working with ZFS on FreeBSD, I now and then I read messages that contain a reference to Illumos which certainly helps to keep up the awareness. Of course there has also been a bit of curiosity – what might the OS be like that grew ZFS? Also the Ravenports project that I participate in planned to support Solaris/Illumos right from the beginning. I wanted to at least be somewhat “prepared” when support for that platform would finally land. So I did a little research on the various derivatives available and settled on the one that I had heard a talk about at last year’s conference of the German Unix Users Group: “OmniOS – Solaris for the Rest of Us”. I would have chosen SmartOS as I admire what Bryan Cantrill does but for getting to know Illumos I prefer a traditional installation over a run-from-RAM system. Of course FreeBSD is not run by corporations, especially when compared to the state of Linux. And when it comes to sponsoring, OpenBSD also takes the money… When it comes to FreeBSD developers, there’s probably some truth to the claim that some of them are using macOS as their desktop systems while OpenBSD devs are more likely to develop on their OS of choice. But then there’s the statement that “every innovation in the past decade comes from Solaris”. Bhyve alone proves this wrong. But let’s be honest: Two of the major technologies that make FreeBSD a great platform today – ZFS and DTrace – actually do come from Solaris. PAM originates there and a more modern way of managing services as well. Also you hear good things about their zones and a lot of small utilities in general. In the end it was a lack of time that made me cheat and go down the easiest road: Create a Vagrantfile and just pull a VM image of the net that someone else had prepared… This worked to just make sure that the Raven packages work on OmniOS. I was determined to return, though – someday. You know how things go: “someday” is a pretty common alias for “probably never, actually.” But then I heard about a forum post on the BSDNow! podcast. The title “Initial OmniOS impressions by a BSD user” caught my attention. I read that it was written by somebody who had used FreeBSD for years but loathed the new Code of Conduct enough to leave. I also oppose the Conduct and have made that pretty clear in my February post [ ! -z ${COC} ] && exit 1. As stated there, I have stayed with my favorite OS and continue to advocate it. I decided to stop reading the post and try things out on my own instead. Now I’ve finally found the time to do so. What’s next? That’s it for part one. In part two I’ll try to make the system useful. So far I have run into a problem that I haven’t been able to solve. But I have some time now to figure things out for the next post. Let’s see if I manage to get it working or if I have to report failure! ###What are all these types of memory in top(1)? Earlier this week I convinced Mark Johnston, one of the FreeBSD VM experts to update a page on the FreeBSD wiki that I saw was being referenced on stackoverflow and similar sites Mark updated the explanations to be more correct, and to include more technical detail for inquiring minds He also added the new type that appeared in FreeBSD somewhat recently Active - Contains memory “actively” (recently) being used by applications Inactive - Contains memory that has not been touched recently, or was released from the Buffer Cache Laundry - Contains memory that Inactive but still potentially contains useful data that needs to be stored before this memory can be used again Wired - Memory that cannot be swapped out, including the kernel, network stack, and the ZFS ARC Buf - Buffer Cache, used my UFS and most filesystems except ZFS (which uses the ARC) Free - Memory that is immediately available for use by the rest of the system ##News Roundup OpenBSD saves me again! — Debug a memory corruption issue Yesterday, I came across a third-part library issue, which crashes at allocating memory: Program terminated with signal SIGSEGV, Segmentation fault. #0 0x00007f594a5a9b6b in _int_malloc () from /usr/lib/libc.so.6 (gdb) bt #0 0x00007f594a5a9b6b in _int_malloc () from /usr/lib/libc.so.6 #1 0x00007f594a5ab503 in malloc () from /usr/lib/libc.so.6 #2 0x00007f594b13f159 in operator new (sz=5767168) at /build/gcc/src/gcc/libstdc++-v3/libsupc++/new_op.cc:50 It is obvious that the memory tags are corrupted, but who is the murder? Since the library involves a lot of maths computation, it is not an easy task to grasp the code quickly. So I need to find another way: (1) Open all warnings during compilation: -Wall. Nothing found. (2) Use valgrind, but unfortunately, valgrind crashes itself: valgrind: the 'impossible' happened: Killed by fatal signal host stacktrace: ==43326== at 0x58053139: get_bszB_as_is (m_mallocfree.c:303) ==43326== by 0x58053139: get_bszB (m_mallocfree.c:315) ==43326== by 0x58053139: vgPlain_arena_malloc (m_mallocfree.c:1799) ==43326== by 0x5800BA84: vgMemCheck_new_block (mc_malloc_wrappers.c:372) ==43326== by 0x5800BD39: vgMemCheck___builtin_vec_new (mc_malloc_wrappers.c:427) ==43326== by 0x5809F785: do_client_request (scheduler.c:1866) ==43326== by 0x5809F785: vgPlain_scheduler (scheduler.c:1433) ==43326== by 0x580AED50: thread_wrapper (syswrap-linux.c:103) ==43326== by 0x580AED50: run_a_thread_NORETURN (syswrap-linux.c:156) sched status: running_tid=1 (3) Change compiler, use clang instead of gcc, and hope it can give me some clues. Still no effect. (4) Switch Operating System from Linux to OpenBSD, the program crashes again. But this time, it tells me where the memory corruption occurs: Program terminated with signal SIGSEGV, Segmentation fault. #0 0x000014b07f01e52d in addMod (r=<error reading variable>, a=4693443247995522, b=28622907746665631, I figure out the issue quickly, and not bother to understand the whole code. OpenBSD saves me again, thanks! ###Native Encryption for ZFS on FreeBSD (Call for Testing) To anyone with an interest in native encryption in ZFS please test the projects/zfs-crypto-merge-0820 branch in my freebsd repo: https://github.com/mattmacy/networking.git git clone https://github.com/mattmacy/networking.git -b projects/zfs-crypto-merge-0820 The UI is quite close to the Oracle Solaris ZFS crypto with minor differences for specifying key location. Please note that once a feature is enabled on a pool it can’t be disabled. This means that if you enable encryption support on a pool you will never be able to import it in to a ZFS without encryption support. For this reason I would strongly advise against using this on any pool that can’t be easily replaced until this change has made its way in to HEAD after the freeze has been lifted. By way of background the original ZoL commit can be found at: https://github.com/zfsonlinux/zfs/pull/5769/commits/5aef9bedc801830264428c64cd2242d1b786fd49 ###VMworld 2018: Showcasing Hybrid Cloud, Persistent Memory and the Asigra TrueNAS Backup Appliance During its last year in Las Vegas before moving back to San Francisco, VMworld was abuzz with all the popular buzzwords, but the key focus was on supporting a more agile approach to hybrid cloud. Surveys of IT stakeholders and analysts agree that most businesses have multiple clouds spanning both public cloud providers and private data centers. While the exact numbers vary, well over half of businesses have a hybrid cloud strategy consisting of at least three different clouds. This focus on hybrid cloud provided the perfect timing for our announcement that iXsystems and Asigra are partnering to deliver the Asigra TrueNAS Backup Appliance, which combines Asigra Cloud Backup software backed by TrueNAS storage. Asigra TrueNAS Backup Appliances provide a self-healing and ransomware-resistent OpenZFS backup repository in your private cloud. The appliance can simultaneously be used as general-purpose file, block, and object storage. How does this tie in with the hybrid cloud? The Asigra Cloud Backup software can backup data from public cloud repositories – G Suite, Office 365, Salesforce, etc. – as well as intelligently move backed-up data to the public cloud for long-term retention. Another major theme at the technical sessions was persistent memory, as vSphere 6.7 added support for persistent memory – either as a storage tier or virtualized and presented to a guest OS. As detailed in our blog post from SNIA’s Persistent Memory Summit 2018, persistent memory is rapidly becoming mainstream. Persistent memory bridges the gap between memory and flash storage – providing near-memory latency storage that persists across reboots or power loss. vSphere allows both legacy and persistent memory-aware applications to leverage this ultra-fast storage tier. We were excited to show off our newly-introduced TrueNAS M-Series at VMworld, as all TrueNAS M40 and M50 models leverage NVDIMM persistent memory technology to provide a super-fast write cache, or SLOG, without any of the limitations of Flash technology. The iXsystems booth’s theme was “Enterprise Storage, Open Source Economics”. iXsystems leverages the power of Open Source software, combined with our enterprise-class hardware and support, to provide incredibly low TCO storage for virtualization environments. Our TrueNAS unified storage and server offerings are an ideal solution for your organization’s private cloud infrastructure. Combined with VMware NSX Hybrid Connect – formerly known as VMware Hybrid Cloud Extension – you can seamlessly shift running systems into a public cloud environment for a true hybrid cloud solution. Another special treat at this year’s booth was iXsystems Vice President of Engineering Kris Moore giving demos of an early version of “Project TrueView”, a single-pane of glass management solution for administration of multiple FreeNAS and TrueNAS systems. In addition to simplified administration and enhanced monitoring, Project TrueView will also provide Role-Based Access Control for finer-grained permissions management. A beta version of Project TrueView is expected to be available at the end of this year. Overall, we had a great week at VMworld 2018 with lots of good conversations with customers, press, analysts, and future customers about TrueNAS, the Asigra TrueNAS Backup Appliance, iXsystems servers, Project TrueView, and more – our booth was more popular than ever! ###End of life for NetBSD 6.x In keeping with NetBSD’s policy of supporting only the latest (8.x) and next most recent (7.x) major branches, the recent release of NetBSD 8.0 marks the end of life for NetBSD 6.x. As in the past, a month of overlapping support has been provided in order to ease the migration to newer releases. As of now, the following branches are no longer maintained: netbsd-6-1 netbsd-6-0 netbsd-6 This means: There will be no more pullups to those branches (even for security issues) There will be no security advisories made for any those branches The existing 6.x releases on ftp.NetBSD.org will be moved into /pub/NetBSD-archive/ May NetBSD 8.0 serve you well! (And if it doesn’t, please submit a PR!) ##Beastie Bits Blast from the past: OpenBSD 3.7 CD artwork People are asking about scale of BSD projects. Let’s figure it out… Tuesday, 21 August 18: me, on ed(1), at SemiBUG arm64 gains RETGUARD Call for participation FreeBSD-UPB/bhyvearm64-utils ##Feedback/Questions Eric - FreeNAS for Vacation Patrick - Long Live Unix Jason - Jason - Full MP3 Recordings Bostjan - Question about jails and kernel Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Episode 262: OpenBSD Surfacing | BSD Now 262
OpenBSD on Microsoft Surface Go, FreeBSD Foundation August Update, What’s taking so long with Project Trident, pkgsrc config file versioning, and MacOS remnants in ZFS code. ##Headlines OpenBSD on the Microsoft Surface Go For some reason I like small laptops and the constraints they place on me (as long as they’re still usable). I used a Dell Mini 9 for a long time back in the netbook days and was recently using an 11" MacBook Air as my primary development machine for many years. Recently Microsoft announced a smaller, cheaper version of its Surface tablets called Surface Go which piqued my interest. Hardware The Surface Go is available in two hardware configurations: one with 4Gb of RAM and a 64Gb eMMC, and another with 8Gb of RAM with a 128Gb NVMe SSD. (I went with the latter.) Both ship with an Intel Pentium Gold 4415Y processor which is not very fast, but it’s certainly usable. The tablet measures 9.65" across, 6.9" tall, and 0.3" thick. Its 10" diagonal 3:2 touchscreen is covered with Gorilla Glass and has a resolution of 1800x1200. The bezel is quite large, especially for such a small screen, but it makes sense on a device that is meant to be held, to avoid accidental screen touches. The keyboard and touchpad are located on a separate, removable slab called the Surface Go Signature Type Cover which is sold separately. I opted for the “cobalt blue” cover which has a soft, cloth-like alcantara material. The cover attaches magnetically along the bottom edge of the device and presents USB-attached keyboard and touchpad devices. When the cover is folded up against the screen, it sends an ACPI sleep signal and is held to the screen magnetically. During normal use, the cover can be positioned flat on a surface or slightly raised up about 3/4" near the screen for better ergonomics. When using the device as a tablet, the cover can be rotated behind the screen which causes it to automatically stop sending keyboard and touchpad events until it is rotated back around. The keyboard has a decent amount of key travel and a good layout, with Home/End/Page Up/Page Down being accessible via Fn+Left/Right/Up/Down but also dedicated Home/End/Page Up/Page Down keys on the F9-F12 keys which I find quite useful since the keyboard layout is somewhat small. By default, the F1-F12 keys do not send F1-F12 key codes and Fn must be used, either held down temporarily or Fn pressed by itself to enable Fn-lock which annoyingly keeps the bright Fn LED illuminated. The keys are backlit with three levels of adjustment, handled by the keyboard itself with the F7 key. The touchpad on the Type Cover is a Windows Precision Touchpad connected via USB HID. It has a decent click feel but when the cover is angled up instead of flat on a surface, it sounds a bit hollow and cheap. Surface Go Pen The touchscreen is powered by an Elantech chip connected via HID-over-i2c, which also supports pen input. A Surface Pen digitizer is available separately from Microsoft and comes in the same colors as the Type Covers. The pen works without any pairing necessary, though the top button on it works over Bluetooth so it requires pairing to use. Either way, the pen requires an AAAA battery inside it to operate. The Surface Pen can attach magnetically to the left side of the screen when not in use. A kickstand can swing out behind the display to use the tablet in a laptop form factor, which can adjust to any angle up to about 170 degrees. The kickstand stays firmly in place wherever it is positioned, which also means it requires a bit of force to pull it out when initially placing the Surface Go on a desk. Along the top of the display are a power button and physical volume rocker buttons. Along the right side are the 3.5mm headphone jack, USB-C port, power port, and microSD card slot located behind the kickstand. Charging can be done via USB-C or the dedicated charge port, which accommodates a magnetically-attached, thin barrel similar to Apple’s first generation MagSafe adapter. The charging cable has a white LED that glows when connected, which is kind of annoying since it’s near the mid-line of the screen rather than down by the keyboard. Unlike Apple’s MagSafe, the indicator light does not indicate whether the battery is charged or not. The barrel charger plug can be placed up or down, but in either direction I find it puts an awkward strain on the power cable coming out of it due to the vertical position of the port. Wireless connectivity is provided by a Qualcomm Atheros QCA6174 802.11ac chip which also provides Bluetooth connectivity. Most of the sensors on the device such as the gyroscope and ambient light sensor are connected behind an Intel Sensor Hub PCI device, which provides some power savings as the host CPU doesn’t have to poll the sensors all the time. Firmware The Surface Go’s BIOS/firmware menu can be entered by holding down the Volume Up button, then pressing and releasing the Power button, and releasing Volume Up when the menu appears. Secure Boot as well as various hardware components can be disabled in this menu. Boot order can also be adjusted. A temporary boot menu can be brought up the same way but using Volume Down instead. ###FreeBSD Foundation Update, August 2018 MESSAGE FROM THE EXECUTIVE DIRECTOR Dear FreeBSD Community Member, It’s been a busy summer for the Foundation. From traveling around the globe spreading the word about FreeBSD to bringing on new team members to improve the Project’s Continuous Integration work, we’re very excited about what we’ve accomplished. Take a minute to check out the latest updates within our Foundation sponsored projects; read more about our advocacy efforts in Bangladesh and community building in Cambridge; don’t miss upcoming Travel Grant deadlines, and new Developer Summits; and be sure to find out how your support will ensure our progress continues into 2019. We can’t do this without you! Happy reading!! Deb August 2018 Development Projects Update Fundraising Update: Supporting the Project August 2018 Release Engineering Update BSDCam 2018 Recap October 2018 FreeBSD Developer Summit Call for Participation SANOG32 and COSCUP 2018 Recap MeetBSD 2018 Travel Grant Application Deadline: September 7 ##News Roundup Project Trident: What’s taking so long? What is taking so long? The short answer is that it’s complicated. Project Trident is quite literally a test of the new TrueOS build system. As expected, there have been quite a few bugs, undocumented features, and other optional bits that we discovered we needed that were not initially present. All of these things have to be addressed and retested in a constant back and forth process. While Ken and JT are both experienced developers, neither has done this kind of release engineering before. JT has done some release engineering back in his Linux days, but the TrueOS and FreeBSD build system is very different. Both Ken and JT are learning a completely new way of building a FreeBSD/TrueOS distribution. Please keep in mind that no one has used this new TrueOS build system before, so Ken and JT want to not only provide a good Trident release, but also provide a model or template for other potential TrueOS distributions too! Where are we now? Through perseverance, trial and error, and a lot of head-scratching we have reached the point of having successful builds. It took a while to get there, but now we are simply working out a few bugs with the new installer that Ken wrote as well as finding and fixing all the new Xorg configuration options which recently landed in FreeBSD. We also found that a number of services have been removed or replaced between TrueOS 18.03 and 18.06 so we are needing to adjust what we consider the “base” services for the desktop. All of these issues are being resolved and we are continually rebuilding and pulling in new patches from TrueOS as soon as they are committed. In the meantime we have made an early BETA release of Trident available to the users in our Telegram Channel for those who want to help out in testing these early versions. Do you foresee any other delays? At the moment we are doing many iterations of testing and tweaking the install ISO and package configurations in order to ensure that all the critical functionality works out-of-box (networking, sound, video, basic apps, etc). While we do not foresee any other major delays, sometimes things happen that our outside of our control. For an example, one of the recent delays that hit recently was completely unexpected: we had a hard drive failure on our build server. Up until recently, The aptly named “Poseidon” build server was running a Micron m500dc drive, but that drive is now constantly reporting errors. Despite ordering a replacement Western Digital Blue SSD several weeks ago, we just received it this past week. The drive is now installed with the builder back to full functionality, but we did lose many precious days with the delay. The build server for Project Trident is very similar to the one that JT donated to the TrueOS project. JT had another DL580 G7, so he donated one to the Trident Project for their build server. Poseidon also has 256GB RAM (64 x 4GB sticks) which is a smidge higher than what the TrueOS builder has. Since we are talking about hardware, we probably should address another question we get often, “What Hardware are the devs testing on?” So let’s go ahead and answer that one now. Developer Hardware JT: His main test box is a custom-built Intel i7 7700K system running 32GB RAM, dual Intel Optane 900P drives, and an Nvidia 1070 GTX with four 4K Acer Monitors. He also uses a Lenovo x250 ThinkPad alongside a desk full of x230t and x220 ThinkPads. One of which he gave away at SouthEast LinuxFest this year, which you can read about here. However it’s not done there, being a complete hardware hoarder, JT also tests on several Intel NUCs and his second laptop a Fujitsu t904, not to mention a Plethora of HP DL580 servers, a DL980 server, and a stack of BL485c, BL460c, and BL490c Blades in his HP c7000 and c3000 Bladecenter chassis. (Maybe it’s time for an intervention for his hardware collecting habits) Ken: For a laptop, he primarily uses a 3rd generation X1 Carbon, but also has an old Eee PC T101MT Netbook (dual core 1GHz, 2GB of memory) which he uses for verifying how well Trident works on low-end hardware. As far as workstations go, his office computer is an Intel i7 with an NVIDIA Geforce GTX 960 running three 4K monitors and he has a couple other custom-built workstations (1 AMD, 1 Intel+NVIDIA) at his home. Generally he assembled random workstations based on hardware that was given to him or that he could acquire cheap. Tim: is using a third gen X1 Carbon and a custom built desktop with an Intel Core i5-4440 CPU, 16 GiB RAM, Nvidia GeForce GTX 750 Ti, and a RealTek 8168 / 8111 network card. Rod: Rod uses… No one knows what Rod uses, It’s kinda like how many licks does it take to get to the center of a Tootsie-Roll Tootsie-Pop… the world may just never know. ###NetBSD GSoC: pkgsrc config file versioning A series of reports from the course of the summer on this Google Summer of Code project The goal of the project is to integrate with a VCS (Version Control System) to make managing local changes to config files for packages easier GSoC 2018 Reports: Configuration files versioning in pkgsrc, Part 1 Packages may install code (both machine executable code and interpreted programs), documentation and manual pages, source headers, shared libraries and other resources such as graphic elements, sounds, fonts, document templates, translations and configuration files, or a combination of them. Configuration files are usually the means through which the behaviour of software without a user interface is specified. This covers parts of the operating systems, network daemons and programs in general that don’t come with an interactive graphical or textual interface as the principal mean for setting options. System wide configuration for operating system software tends to be kept under /etc, while configuration for software installed via pkgsrc ends up under LOCALBASE/etc (e.g., /usr/pkg/etc). Software packaged as part of pkgsrc provides example configuration files, if any, which usually get extracted to LOCALBASE/share/examples/PKGBASE/. Don’t worry: automatic merging is disabled by default, set $VCSAUTOMERGE to enable it. In order to avoid breakage, installed configuration is backed up first in the VCS, separating user-modified files from files that have been already automatically merged in the past, in order to allow the administrator to easily restore the last manually edited file in case of breakage. VCS functionality only applies to configuration files, not to rc.d scripts, and only if the environment variable $NOVCS is unset. The version control system to be used as a backend can be set through $VCS. It default to RCS, the Revision Control System, which works only locally and doesn’t support atomic transactions. Other backends such as CVS are supported and more will come; these, being used at the explicit request of the administrator, need to be already installed and placed in a directory part of $PATH. GSoC 2018 Reports: Configuration files versioning in pkgsrc, part 2: remote repositories (git and CVS) pkgsrc is now able to deploy configuration from packages being installed from a remote, site-specific vcs repository. User modified files are always tracked even if automerge functionality is not enabled, and a new tool, pkgconftrack(1), exists to manually store user changes made outside of package upgrade time. Version Control software is executed as the same user running pkgadd or make install, unless the user is “root”. In this case, a separate, unprivileged user, pkgvcsconf, gets created with its own home directory and a working login shell (but no password). The home directory is not strictly necessary, it exists to facilitate migrations betweens repositories and vcs changes; it also serves to store keys used to access remote repositories. Using git instead of rcs is simply done by setting VCS=git in pkginstall.conf GSoC 2018 Reports: Configuration files versioning in pkgsrc, part 3: remote repositories (SVN and Mercurial) GSoC 2018 Reports: Configuration files versioning in pkgsrc, part 4: configuration deployment, pkgtools and future improvements Support for configuration tracking is in scripts, pkginstall scripts, that get built into binary packages and are run by pkgadd upon installation. The idea behind the proposal suggested that users of the new feature should be able to store revisions of their installed configuration files, and of package-provided default, both in local or remote repositories. With this capability in place, it doesn’t take much to make the scripts “pull” configuration from a VCS repository at installation time. That’s what setting VCSCONFPULL=yes in pkginstall.conf after having enabled VCSTRACKCONF does: You are free to use official, third party prebuilt packages that have no customization in them, enable these options, and point pkgsrc to a private conf repository. If it contains custom configuration for the software you are installing, an attempt will be made to use it and install it on your system. If it fails, pkginstall will fall back to using the defaults that come inside the package. RC scripts are always deployed from the binary package, if existing and PKGRCDSCRIPTS=yes in pkginstall.conf or the environment. This will be part of packages, not a separate solution like configuration management tools. It doesn’t support running scripts on the target system to customize the installation, it doesn’t come with its domain-specific language, it won’t run as a daemon or require remote logins to work. It’s quite limited in scope, but you can define a ROLE for your system in pkginstall.conf or in the environment, and pkgsrc will look for configuration you or your organization crafted for such a role (e.g., public, standalone webserver vs reverse proxy or node in a database cluster) ###A little bit of the one-time MacOS version still lingers in ZFS Once upon a time, Apple came very close to releasing ZFS as part of MacOS. Apple did this work in its own copy of the ZFS source base (as far as I know), but the people in Sun knew about it and it turns out that even today there is one little lingering sign of this hoped-for and perhaps prepared-for ZFS port in the ZFS source code. Well, sort of, because it’s not quite in code. Lurking in the function that reads ZFS directories to turn (ZFS) directory entries into the filesystem independent format that the kernel wants is the following comment: objnum = ZFSDIRENTOBJ(zap.zafirstinteger); / MacOS X can extract the object type here such as: * uint8t type = ZFSDIRENTTYPE(zap.zafirstinteger); */ Specifically, this is in zfsreaddir in zfsvnops.c . ZFS maintains file type information in directories. This information can’t be used on Solaris (and thus Illumos), where the overall kernel doesn’t have this in its filesystem independent directory entry format, but it could have been on MacOS (‘Darwin’), because MacOS is among the Unixes that support d_type. The comment itself dates all the way back to this 2007 commit, which includes the change ‘reserve bits in directory entry for file type’, which created the whole setup for this. I don’t know if this file type support was added specifically to help out Apple’s MacOS X port of ZFS, but it’s certainly possible, and in 2007 it seems likely that this port was at least on the minds of ZFS developers. It’s interesting but understandable that FreeBSD didn’t seem to have influenced them in the same way, at least as far as comments in the source code go; this file type support is equally useful for FreeBSD, and the FreeBSD ZFS port dates to 2007 too (per this announcement). Regardless of the exact reason that ZFS picked up maintaining file type information in directory entries, it’s quite useful for people on both FreeBSD and Linux that it does so. File type information is useful for any number of things and ZFS filesystems can (and do) provide this information on those Unixes, which helps make ZFS feel like a truly first class filesystem, one that supports all of the expected general system features. ##Beastie Bits Mac-like FreeBSD Laptop Syncthing on FreeBSD New ZFS Boot Environments Tool My system’s time was so wrong, that even ntpd didn’t work OpenSSH 7.8/7.8p1 (2018-08-24) EuroBSD (Sept 20-23rd) registration Early Bird Period is coming to an end MeetBSD (Oct 18-20th) is coming up fast, hurry up and register! AsiaBSDcon 2019 Dates ##Feedback/Questions Will - Kudos and a Question Peter - Fanless Computers Ron - ZFS disk clone or replace or something Bostjan - ZFS Record Size Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Episode 261: FreeBSDcon Flashback | BSD Now 261
Insight into TrueOS and Trident, stop evildoers with pf-badhost, Flashback to FreeBSDcon ‘99, OpenBSD’s measures against TLBleed, play Morrowind on OpenBSD in 5 steps, DragonflyBSD developers shocked at Threadripper performance, and more. ##Headlines An Insight into the Future of TrueOS BSD and Project Trident Last month, TrueOS announced that they would be spinning off their desktop offering. The team behind the new project, named Project Trident, have been working furiously towards their first release. They did take a few minutes to answer some of our question about Project Trident and TrueOS. I would like to thank JT and Ken for taking the time to compile these answers. It’s FOSS: What is Project Trident? Project Trident: Project Trident is the continuation of the TrueOS Desktop. Essentially, it is the continuation of the primary “TrueOS software” that people have been using for the past 2 years. The continuing evolution of the entire TrueOS project has reached a stage where it became necessary to reorganize the project. To understand this change, it is important to know the history of the TrueOS project. Originally, Kris Moore created PC-BSD. This was a Desktop release of FreeBSD focused on providing a simple and user-friendly graphical experience for FreeBSD. PC-BSD grew and matured over many years. During the evolution of PC-BSD, many users began asking for a server focused version of the software. Kris agreed, and TrueOS was born as a scaled down server version of PC-BSD. In late 2016, more contributors and growth resulted in significant changes to the PC-BSD codebase. Because the new development was so markedly different from the original PC-BSD design, it was decided to rebrand the project. TrueOS was chosen as the name for this new direction for PC-BSD as the project had grown beyond providing only a graphical front to FreeBSD and was beginning to make fundamental changes to the FreeBSD operating system. One of these changes was moving PC-BSD from being based on each FreeBSD Release to TrueOS being based on the active and less outdated FreeBSD Current. Other major changes are using OpenRC for service management and being more aggressive about addressing long-standing issues with the FreeBSD release process. TrueOS moved toward a rolling release cycle, twice a year, which tested and merged FreeBSD changes directly from the developer instead of waiting months or even years for the FreeBSD review process to finish. TrueOS also deprecated and removed obsolete technology much more regularly. As the TrueOS Project grew, the developers found these changes were needed by other FreeBSD-based projects. These projects began expressing interest in using TrueOS rather than FreeBSD as the base for their project. This demonstrated that TrueOS needed to again evolve into a distribution framework for any BSD project to use. This allows port maintainers and source developers from any BSD project to pool their resources and use the same source repositories while allowing every distribution to still customize, build, and release their own self-contained project. The result is a natural split of the traditional TrueOS team. There were now naturally two teams in the TrueOS project: those working on the build infrastructure and FreeBSD enhancements – the “core” part of the project, and those working on end-user experience and utility – the “desktop” part of the project. When the decision was made to formally split the projects, the obvious question that arose was what to call the “Desktop” project. As TrueOS was already positioned to be a BSD distribution platform, the developers agreed the desktop side should pick a new name. There were other considerations too, one notable being that we were concerned that if we continued to call the desktop project “TrueOS Desktop”, it would prevent people from considering TrueOS as the basis for their distribution because of misconceptions that TrueOS was a desktop-focused OS. It also helps to “level the playing field” for other desktop distributions like GhostBSD so that TrueOS is not viewed as having a single “blessed” desktop version. It’s FOSS: What features will TrueOS add to the FreeBSD base? Project Trident: TrueOS has already added a number of features to FreeBSD: OpenRC replaces rc.d for service management LibreSSL in base Root NSS certificates out-of-box Scriptable installations (pc-sysinstall) The full list of changes can be seen on the TrueOS repository (https://github.com/trueos/trueos/blob/trueos-master/README.md). This list does change quite regularly as FreeBSD development itself changes. It’s FOSS: I understand that TrueOS will have a new feature that will make creating a desktop spin of TrueOS very easy. Could you explain that new feature? Project Trident: Historically, one of the biggest hurdles for creating a desktop version of FreeBSD is that the build options for packages are tuned for servers rather than desktops. This means a desktop distribution cannot use the pre-built packages from FreeBSD and must build, use, and maintain a custom package repository. Maintaining a fork of the FreeBSD ports tree is no trivial task. TrueOS has created a full distribution framework so now all it takes to create a custom build of FreeBSD is a single JSON manifest file. There is now a single “source of truth” for the source and ports repositories that is maintained by the TrueOS team and regularly tagged with “stable” build markers. All projects can use this framework, which makes updates trivial. It’s FOSS: Do you think that the new focus of TrueOS will lead to the creation of more desktop-centered BSDs? Project Trident: That is the hope. Historically, creating a desktop-centered BSD has required a lot of specialized knowledge. Not only do most people not have this knowledge, but many do not even know what they need to learn until they start troubleshooting. TrueOS is trying to drastically simplify this process to enable the wider Open Source community to experiment, contribute, and enjoy BSD-based projects. It’s FOSS: What is going to happen to TrueOS Pico? Will Project Trident have ARM support? Project Trident: Project Trident will be dependent on TrueOS for ARM support. The developers have talked about the possibility of supporting ARM64 and RISC-V architectures, but it is not possible at the current time. If more Open Source contributors want to help develop ARM and RISC-V support, the TrueOS project is definitely willing to help test and integrate that code. It’s FOSS: What does this change (splitting Trus OS into Project Trident) mean for the Lumina desktop environment? Project Trident: Long-term, almost nothing. Lumina is still the desktop environment for Project Trident and will continue to be developed and enhanced alongside Project Trident just as it was for TrueOS. Short-term, we will be delaying the release of Lumina 2.0 and will release an updated version of the 1.x branch (1.5.0) instead. This is simply due to all the extra overhead to get Project Trident up and running. When things settle down into a rhythm, the development of Lumina will pick up once again. It’s FOSS: Are you planning on including any desktop environments besides Lumina? Project Trident: While Lumina is included by default, all of the other popular desktop environments will be available in the package repo exactly as they had been before. It’s FOSS: Any plans to include Steam to increase the userbase? Project Trident: Steam is still unavailable natively on FreeBSD, so we do not have any plans to ship it out of the box currently. In the meantime, we highly recommend installing the Windows version of Steam through the PlayOnBSD utility. It’s FOSS: What will happen to the AppCafe? Project Trident: The AppCafe is the name of the graphical interface for the “pkg” utility integrated into the SysAdm client created by TrueOS. This hasn’t changed. SysAdm, the graphical client, and by extension AppCafe are still available for all TrueOS-based distributions to use. It’s FOSS: Does Project Trident have any corporate sponsors lined up? If not, would you be open to it or would you prefer that it be community supported? Project Trident: iXsystems is the first corporate sponsor of Project Trident and we are always open to other sponsorships as well. We would prefer smaller individual contributions from the community, but we understand that larger project needs or special-purpose goals are much more difficult to achieve without allowing larger corporate sponsorships as well. In either case, Project Trident is always looking out for the best interests of the community and will not allow intrusive or harmful code to enter the project even if a company or individual tries to make that code part of a sponsorship deal. It’s FOSS: BSD always seems to be lagging in terms of support for newer devices. Will TrueOS be able to remedy that with a quicker release cycle? Project Trident: Yes! That was a primary reason for TrueOS to start tracking the CURRENT branch of FreeBSD in 2016. This allows for the changes that FreeBSD developers are making, including new hardware support, to be available much sooner than if we followed the FreeBSD release cycle. It’s FOSS: Do you have any idea when Project Trident will have its first release? Project Trident: Right now we are targeting a late August release date. This is because Project Trident is “kicking the wheels” on the new TrueOS distribution system. We want to ensure everything is working smoothly before we release. Going forward, we plan on having regular package updates every week or two for the end-user packages and a new release of Trident with an updated OS version every 6 months. This will follow the TrueOS release schedule with a small time offset. ###pf-badhost: Stop the evil doers in their tracks! pf-badhost is a simple, easy to use badhost blocker that uses the power of the pf firewall to block many of the internet’s biggest irritants. Annoyances such as ssh bruteforcers are largely eliminated. Shodan scans and bots looking for webservers to abuse are stopped dead in their tracks. When used to filter outbound traffic, pf-badhost blocks many seedy, spooky malware containing and/or compromised webhosts. Filtering performance is exceptional, as the badhost list is stored in a pf table. To quote the OpenBSD FAQ page regarding tables: “the lookup time on a table holding 50,000 addresses is only slightly more than for one holding 50 addresses.” pf-badhost is simple and powerful. The blocklists are pulled from quality, trusted sources. The ‘Firehol’, ‘Emerging Threats’ and ‘Binary Defense’ block lists are used as they are popular, regularly updated lists of the internet’s most egregious offenders. The pf-badhost.sh script can easily be expanded to use additional or alternate blocklists. pf-badhost works best when used in conjunction with unbound-adblock for the ultimate badhost blocking. Notes: If you are trying to run pf-badhost on a LAN or are using NAT, you will want to add a rule to your pf.conf appearing BEFORE the pf-badhost rules allowing traffic to and from your local subnet so that you can still access your gateway and any DNS servers. Conversely, adding a line to pf-badhost.sh that removes your subnet range from the <pfbadhost> table should also work. Just make sure you choose a subnet range / CIDR block that is actually in the list. 192.168.0.0/16, 172.16.0.0/12 and 10.0.0.0/8 are the most common home/office subnet ranges. DigitalOcean https://do.co/bsdnow ###FLASHBACK: FreeBSDCon’99: Fans of Linux’s lesser-known sibling gather for the first time FreeBSD, a port of BSD Unix to Intel, has been around almost as long as Linux has – but without the media hype. Its developer and user community recently got a chance to get together for the first time, and they did it in the city where BSD – the Berkeley Software Distribution – was born some 25 years ago. October 17, 1999 marked a milestone in the history of FreeBSD – the first FreeBSD conference was held in the city where it all began, Berkeley, CA. Over 300 developers, users, and interested parties attended from around the globe. This was easily 50 percent more people than the conference organizers had expected. This first conference was meant to be a gathering mostly for developers and FreeBSD advocates. The turnout was surprisingly (and gratifyingly) large. In fact, attendance exceeded expectations so much that, for instance, Kirk McKusick had to add a second, identical tutorial on FreeBSD internals, because it was impossible for everyone to attend the first! But for a first-ever conference, I was impressed by how smoothly everything seemed to go. Sessions started on time, and the sessions I attended were well-run; nothing seemed to be too cold, dark, loud, late, or off-center. Of course, the best part about a conference such as this one is the opportunity to meet with other people who share similar interests. Lunches and breaks were a good time to meet people, as was the Tuesday night beer bash. The Wednesday night reception was of a type unusual for the technical conferences I usually attend – a three-hour Hornblower dinner cruise on San Francisco Bay. Not only did we all enjoy excellent food and company, but we all got to go up on deck and watch the lights of San Francisco and Berkeley as we drifted by. Although it’s nice when a conference attracts thousands of attendees, there are some things that can only be done with smaller groups of people; this was one of them. In short, this was a tiny conference, but a well-run one. Sessions Although it was a relatively small conference, the number and quality of the sessions belied the size. Each of the three days of the conference featured a different keynote speaker. In addition to Jordan Hubbard, Jeremy Allison spoke on “Samba Futures” on day two, and Brian Behlendorf gave a talk on “FreeBSD and Apache: A Perfect Combo” to start off the third day. The conference sessions themselves were divided into six tracks: advocacy, business, development, networking, security, and panels. The panels track featured three different panels, made up of three different slices of the community: the FreeBSD core team, a press panel, and a prominent user panel with representatives from such prominent commercial users as Yahoo! and USWest. I was especially interested in Apple Computer’s talk in the development track. Wilfredo Sanchez, technical lead for open source projects at Apple (no, that’s not an oxymoron!) spoke about Apple’s Darwin project, the company’s operating system road map, and the role of BSD (and, specifically, FreeBSD) in Apple’s plans. Apple and Unix have had a long and uneasy history, from the Lisa through the A/UX project to today. Personally, I’m very optimistic about the chances for the Darwin project to succeed. Apple’s core OS kernel team has chosen FreeBSD as its reference platform. I’m looking forward to what this partnership will bring to both sides. Other development track sessions included in-depth tutorials on writing device drivers, basics of the Vinum Volume Manager, Fibre Channel, development models (the open repository model), and the FreeBSD Documentation Project (FDP). If you’re interested in contributing to the FreeBSD project, the FDP is a good place to start. Advocacy sessions included “How One Person Can Make a Difference” (a timeless topic that would find a home at any technical conference!) and “Starting and Managing A User Group” (trials and tribulations as well as rewards). The business track featured speakers from three commercial users of FreeBSD: Cybernet, USWest, and Applix. Applix presented its port of Applixware Office for FreeBSD and explained how Applix has taken the core services of Applixware into open source. Commercial applications and open source were once a rare combination; we can only hope the trend away from that state of affairs will continue. Commercial use of FreeBSD The use of FreeBSD in embedded applications is increasing as well – and it is increasing at the same rate that hardware power is. These days, even inexpensive systems are able to run a BSD kernel. The BSD license and the solid TCP/IP stack prove significant enticements to this market as well. (Unlike the GNU Public License, the BSD license does not require that vendors make derivative works open source.) Companies such as USWest and Verio use FreeBSD for a wide variety of different Internet services. Yahoo! and Hotmail are examples of companies that use FreeBSD extensively for more specific purposes. Yahoo!, for example, has many hundreds of FreeBSD boxes, and Hotmail has almost 2000 FreeBSD machines at its data center in the San Francisco Bay area. Hotmail is owned by Microsoft, so the fact that it runs FreeBSD is a secret. Don’t tell anyone… When asked to comment on the increasing commercial interest in BSD, Hubbard said that FreeBSD is learning the Red Hat lesson. “Walnut Creek and others with business interests in FreeBSD have learned a few things from the Red Hat IPO,” he said, “and nobody is just sitting around now, content with business as usual. It’s clearly business as unusual in the open source world today.” Hubbard had also singled out some of BSD’s commercial partners, such as Whistle Communications, for praise in his opening day keynote. These partners play a key role in moving the project forward, he said, by contributing various enhancements and major new systems, such as Netgraph, as well as by contributing paid employee time spent on FreeBSD. Even short FreeBSD-related contacts can yield good results, Hubbard said. An example of this is the new jail() security code introduced in FreeBSD 3.x and 4.0, which was contributed by R & D Associates. A number of ISPs are also now donating the hardware and bandwidth that allows the project to provide more resource mirrors and experimental development sites. See you next year And speaking of corporate sponsors, thanks go to Walnut Creek for sponsoring the conference, and to Yahoo! for covering all the expenses involved in bringing the entire FreeBSD core team to Berkeley. As a fan of FreeBSD, I’m happy to see that the project has finally produced a conference. It was time: many of the 16 core team members had been working together on a regular basis for nearly seven years without actually meeting face to face. It’s been an interesting year for open source projects. I’m looking forward to the next year – and the next BSD conference – to be even better. ##News Roundup OpenBSD Recommends: Disable SMT/Hyperthreading in all Intel BIOSes Two recently disclosed hardware bugs affected Intel cpus: - TLBleed - T1TF (the name "Foreshadow" refers to 1 of 3 aspects of this bug, more aspects are surely on the way) Solving these bugs requires new cpu microcode, a coding workaround, *AND* the disabling of SMT / Hyperthreading. SMT is fundamentally broken because it shares resources between the two cpu instances and those shared resources lack security differentiators. Some of these side channel attacks aren't trivial, but we can expect most of them to eventually work and leak kernel or cross-VM memory in common usage circumstances, even such as javascript directly in a browser. There will be more hardware bugs and artifacts disclosed. Due to the way SMT interacts with speculative execution on Intel cpus, I expect SMT to exacerbate most of the future problems. A few months back, I urged people to disable hyperthreading on all Intel cpus. I need to repeat that: DISABLE HYPERTHREADING ON ALL YOUR INTEL MACHINES IN THE BIOS. Also, update your BIOS firmware, if you can. OpenBSD -current (and therefore 6.4) will not use hyperthreading if it is enabled, and will update the cpu microcode if possible. But what about 6.2 and 6.3? The situation is very complex, continually evolving, and is taking too much manpower away from other tasks. Furthermore, Intel isn't telling us what is coming next, and are doing a terrible job by not publically documenting what operating systems must do to resolve the problems. We are having to do research by reading other operating systems. There is no time left to backport the changes -- we will not be issuing a complete set of errata and syspatches against 6.2 and 6.3 because it is turning into a distraction. Rather than working on every required patch for 6.2/6.3, we will re-focus manpower and make sure 6.4 contains the best solutions possible. So please try take responsibility for your own machines: Disable SMT in the BIOS menu, and upgrade your BIOS if you can. I'm going to spend my money at a more trustworthy vendor in the future. ###Get Morrowind running on OpenBSD in 5 simple steps This article contains brief instructions on how to get one of the greatest Western RPGs of all time, The Elder Scrolls III: Morrowind, running on OpenBSD using the OpenMW open source engine recreation. These instructions were tested on a ThinkPad X1 Carbon Gen 3. The information was adapted from this OpenMW forum thread: https://forum.openmw.org/viewtopic.php?t=3510 Purchase and download the DRM-free version from GOG (also considered the best version due to the high quality PDF guide that it comes with): https://www.gog.com/game/theelderscrollsiiimorrowindgotyedition Install the required packages built from the ports tree as root. openmw is the recreated game engine, and innoextract is how we will get the game data files out of the win32 executable. pkgadd openmw innoextract Move the file from GOG setuptesmorrowindgoty2.0.0.7.exe into its own directory morrowind/ due to innoextract’s default behaviour of extracting into the current directory. Then type: innoextract setuptesmorrowindgoty2.0.0.7.exe Type openmw-wizard and follow the straightforward instructions. Note that you have a pre-existing installation, and select the morrowind/app/Data Files folder that innoextract extracted. Type in openmw-launcher, toggle the settings to your preferences, and then hit play! iXsystems https://twitter.com/allanjude/status/1034647571124367360 ###My First Clang Bug Part of the role of being a packager is compiling lots (and lots) of packages. That means compiling lots of code from interesting places and in a variety of styles. In my opinion, being a good packager also means providing feedback to upstream when things are bad. That means filing upstream bugs when possible, and upstreaming patches. One of the “exciting” moments in packaging is when tools change. So each and every major CMake update is an exercise in recompiling 2400 or more packages and adjusting bits and pieces. When a software project was last released in 2013, adjusting it to modern tools can become quite a chore (e.g. Squid Report Generator). CMake is excellent for maintaining backwards compatibility, generally accommodating old software with new policies. The most recent 3.12 release candidate had three issues filed from the FreeBSD side, all from fallout with older software. I consider the hours put into good bug reports, part of being a good citizen of the Free Software world. My most interesting bug this week, though, came from one line of code somewhere in Kleopatra: QUNUSED(gpgagentdata); That one line triggered a really peculiar link error in KDE’s FreeBSD CI system. Yup … telling the compiler something is unused made it fall over. Commenting out that line got rid of the link error, but introduced a warning about an unused function. Working with KDE-PIM’s Volker Krause, we whittled the problem down to a six-line example program — two lines if you don’t care much for coding style. I’m glad, at that point, that I could throw it over the hedge to the LLVM team with some explanatory text. Watching the process on their side reminds me ever-so-strongly of how things work in KDE (or FreeBSD for that matter): Bugzilla, Phabricator, and git combine to be an effective workflow for developers (perhaps less so for end-users). Today I got a note saying that the issue had been resolved. So brief a time for a bug. Live fast. Get squashed young. ###DragonFlyBSD Now Runs On The Threadripper 2990WX, Developer Shocked At Performance Last week I carried out some tests of BSD vs. Linux on the new 32-core / 64-thread Threadripper 2990WX. I tested FreeBSD 11, FreeBSD 12, and TrueOS – those benchmarks will be published in the next few days. I tried DragonFlyBSD, but at the time it wouldn’t boot with this AMD HEDT processor. But now the latest DragonFlyBSD development kernel can handle the 2990WX and the lead DragonFly developer calls this new processor “a real beast” and is stunned by its performance potential. When I tried last week, the DragonFlyBSD 5.2.2 stable release nor DragonFlyBSD 5.3 daily snapshot would boot on the 2990WX. But it turns out Matthew Dillon, the lead developer of DragonFlyBSD, picked up a rig and has it running now. So in time for the next 5.4 stable release or those using the daily snapshots can have this 32-core / 64-thread Zen+ CPU running on this operating system long ago forked from FreeBSD. In announcing his success in bringing up the 2990WX under DragonFlyBSD, which required a few minor changes, he shared his performance thoughts and hopes for the rig. “The cpu is a real beast, packing 32 cores and 64 threads. It blows away our dual-core Xeon to the tune of being +50% faster in concurrent compile tests, and it also blows away our older 4-socket Opteron (which we call ‘Monster’) by about the same margin. It’s an impressive CPU. For now the new beast is going to be used to help us improve I/O performance through the filesystem, further SMP work (but DFly scales pretty well to 64 threads already), and perhaps some driver to work to support the 10gbe on the mobo.” Dillon shared some results on the system as well. " The Threadripper 2990WX is a beast. It is at least 50% faster than both the quad socket opteron and the dual socket Xeon system I tested against. The primary limitation for the 2990WX is likely its 4 channels of DDR4 memory, and like all Zen and Zen+ CPUs, memory performance matters more than CPU frequency (and costs almost no power to pump up the performance). That said, it still blow away a dual-socket Xeon with 3x the number of memory channels. That is impressive!" The well known BSD developer also added, “This puts the 2990WX at par efficiency vs a dual-socket Xeon system, and better than the dual-socket Xeon with slower memory and a power cap. This is VERY impressive. I should note that the 2990WX is more specialized with its asymetric NUMA architecture and 32 cores. I think the sweet spot in terms of CPU pricing and efficiency is likely going to be with the 2950X (16-cores/32-threads). It is clear that the 2990WX (32-cores/64-threads) will max out 4-channel memory bandwidth for many workloads, making it a more specialized part. But still awesome…This thing is an incredible beast, I’m glad I got it.” While I have the FreeBSD vs. Linux benchmarks from a few days ago, it looks like now on my ever growing TODO list will be re-trying out the newest DragonFlyBSD daily snapshot for seeing how the performance compares in the mix. Stay tuned for the numbers that should be in the next day or two. ##Beastie Bits X11 on really small devices mandoc-1.14.4 released The pfSense Book is now available to everyone MWL: Burn it down! Burn it all down! Configuring OpenBSD: System and user config files for a more pleasant laptop FreeBSD Security Advisory: Resource exhaustion in TCP reassembly OpenBSD Foundation gets first 2018 Iridium donation New ZFS commit solves issue a few users reported in the feedback segment Project Trident should have a beta release by the end of next week Reminder about Stockholm BUG: September 5, 17:30-22:00 BSD-PL User Group: September 13, 18:30-21:00 Tarsnap ##Feedback/Questions Malcom - Having different routes per interface Bostjan - ZFS and integrity of data Michael - Suggestion for Monitoring Barry - Feedback Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Episode 260: Hacking Tour of Europe | BSD Now 260
Trip reports from the Essen Hackathon and BSDCam, CfT: ZFS native encryption and UFS trim consolidation, ZFS performance benchmarks on a FreeBSD server, how to port your OS to EC2, Vint Cerf about traceability, Remote Access console to an RPi3 running FreeBSD, and more. ##Headlines Essen Hackathon & BSDCam 2018 trip report Allan and Benedict met at FRA airport and then headed to the Air Rail terminal for our train to Essen where the Hackathon would happen over the weekend of Aug 10 - 12, 2018. Once there, we did not have to wait long until other early-arrivals would show up and soon we had about 10 people gathered for lunch. After buying some take-out pizzas and bringing it back to the Linuxhotel (there was a training still going on there so we could not get into our rooms yet), we sat in the sunny park and talked. More and more people arrived and soon, people started hacking on their laptops. Some people would not arrive until a few hours before midnight, but we already had a record appearance of 20 people in total. On Saturday, we gathered everyone in one of the seminar rooms that had rooms and chairs for us. After some organizational infos, we did an introductory round and Benedict wrote down on the whiteboard what people were interested in. It was not long until groups formed to talk about SSL in base, weird ZFS scrubs that would go over 100% completion (fixed now). Other people started working on ports, fixing bugs, or wrote documentation. The day ended in a BBQ in the Linuxhotel park, which was well received by everyone. On Sunday, after attendees packed up their luggage and stored it in the seminar room, we continued hacking until lunchtime. After a quick group picture, we headed to a local restaurant for the social event (which was not open on Saturday, otherwise we would have had it then). In the afternoon, most people departed, a good half of them were heading for BSDCam. Commits from the hackathon (the ones from 2018) Overall, the hackathon was well received by attendees and a lot of them liked the fact that it was close to another BSD gathering so they could nicely combine the two. Also, people thought about doing their own hackathon in the future, which is an exciting prospect. Thanks to all who attended, helped out here and there when needed. Special Thanks to Netzkommune GmbH for sponsoring the social event and the Linuxhotel for having us. Benedict was having a regular work day on Monday after coming back from the hackathon, but flew out to Heathrow on Tuesday. Allan was in London a day earlier and arrived a couple of hours before Benedict in Cambridge. He headed for the Computer Lab even though the main event would not start until Wednesday. Most people gathered at the Maypole pub on Tuesday evening for welcomes, food and drinks. On Wednesday, a lot of people met in the breakfast room of Churchill College where most people were staying and went to the Computer Lab, which served as the main venue for BSDCam, together. The morning was spend with introductions and collecting what most people were interested in talking. This unconference style has worked well in the past and soon we had 10 main sessions together for the rest of this and the following two days (full schedule). Most sessions took notes, which you can find on the FreeBSD wiki. On Thursday evening, we had a nice formal dinner at Trinity Hall. BSDCam 2018 was a great success with a lot of fruitful discussions and planning sessions. We thank the organizers for BSDCam for making it happen. A special mentions goes out to Robert Watson and his family. Even though he was not there, he had a good reason to miss it: they had their first child born at the beginning of the week. Congratulations and best wishes to all three of them! ###Call for Testing: ZFS Native Encryption for FreeBSD A port of the ZoL (ZFS-on-Linux) feature that provides native crypto support for ZFS is ready for testing on FreeBSD Most of the porting was done by sef@freebsd.org (Sean Eric Fagan) The original ZoL commit is here: https://github.com/zfsonlinux/zfs/pull/5769/commits/5aef9bedc801830264428c64cd2242d1b786fd49 For an overview, see Tom Caputi’s presentation from the OpenZFS Developers Summit in 2016 Video: https://youtu.be/frnLiXclAMo Slides: https://drive.google.com/file/d/0B5hUzsxe4cdmU3ZTRXNxa2JIaDQ/view?usp=sharing WARNING: test in VMs or with spare disks etc, pools created with this code, or upgraded to this version, will no longer be importable on systems that do not support this feature. The on-disk format or other things may change before the final version, so you will likely have to ‘zfs send | zfs recv’ the data on to a new pool Thanks for testing to help this feature land in FreeBSD iXsystems ###Call for Testing: UFS TRIM Consolidation Kirk Mckusick posts to the FreeBSD mailing list looking for testers for the new UFS TRIM Consolidation code When deleting files on filesystems that are stored on flash-memory (solid-state) disk drives, the filesystem notifies the underlying disk of the blocks that it is no longer using. The notification allows the drive to avoid saving these blocks when it needs to flash (zero out) one of its flash pages. These notifications of no-longer-being-used blocks are referred to as TRIM notifications. In FreeBSD these TRIM notifications are sent from the filesystem to the drive using the BIODELETE command. Until now, the filesystem would send a separate message to the drive for each block of the file that was deleted. Each Gigabyte of file size resulted in over 3000 TRIM messages being sent to the drive. This burst of messages can overwhelm the drive’s task queue causing multiple second delays for read and write requests. This implementation collects runs of contiguous blocks in the file and then consolodates them into a single BIODELETE command to the drive. The BIODELETE command describes the run of blocks as a single large block being deleted. Each Gigabyte of file size can result in as few as two BIODELETE commands and is typically less than ten. Though these larger BIODELETE commands take longer to run, they do not clog the drive task queue, so read and write commands can intersperse effectively with them. Though this new feature has been throughly reviewed and tested, it is being added disabled by default so as to minimize the possibility of disrupting the upcoming 12.0 release. It can be enabled by running `sysctl vfs.ffs.dotrimcons=1’’. Users are encouraged to test it. If no problems arise, we will consider requesting that it be enabled by default for 12.0. This support is off by default, but I am hoping that I can get enough testing to ensure that it (a) works, and (b) is helpful that it will be reasonable to have it turned on by default in 12.0. The cutoff for turning it on by default in 12.0 is September 19th. So I am requesting your testing feedback in the near-term. Please let me know if you have managed to use it successfully (or not) and also if it provided any performance difference (good or bad). To enable TRIM consolidation usesysctl vfs.ffs.dotrimcons=1’ There is also a diff that adds additional statistics: https://lists.freebsd.org/pipermail/freebsd-current/2018-August/070798.html You can also watch the volume and latency of BIODELETE commands by running gstat with the -d flag ##News Roundup ZFS performance Aravindh Sampathkumar, a Performance Engineer and Sysadmin posts some simple benchmarks he did on a new ZFS server This is NOT an all-in post about ZFS performance. I built a FreeBSD+ZFS file server recently at work to serve as an offsite backup server. I wanted to run a few synthetic workloads on it and look at how it fares from performance perspective. Mostly for curiosity and learning purposes. As stated in the notes about building this server, performance was not one of the priorities, as this server will never face our active workload. What I care about from this server is its ability to work with rsync and keep the data synchronised with our primary storage server. With that context, I ran a few write tests to see how good our solution is and what to expect from it in terms of performance. The article then uses FIO to do some benchmarks. As the author did, make sure you match the FIO block size to the ZFS record size to avoid write amplification. Either tune FIO or adjust the recordsize property in ZFS You also want to consider compression and cache effects Write Performance: Incompressible: 1600-2600 MB/s, Compressible: 2500-6600 MB/s Another over 1200 MB/s is enough to keep your 10 gigabit network saturated The increased latency that is seen with higher number of writers working, may be the result of the ZFS backpressure system (the write throttle). There is some tuning that can be done there. Specifically, since this machine has 768 GB of ram, you might allow more than 4GB of dirty data, which would mean you’d be able to write larger batches and not have to push back while you wait for a transaction group to flush when dealing with gigabytes/sec of writes ###How to port your OS to EC2 Colin Percival reflects on his FreeBSD on EC2 maintainership efforts in his blog: I’ve been the maintainer of the FreeBSD/EC2 platform for about 7.5 years now, and as far as “running things in virtual machines” goes, that remains the only operating system and the only cloud which I work on. That said, from time to time I get questions from people who want to port other operating systems into EC2, and being a member of the open source community, I do my best to help them. I realized a few days ago that rather than replying to emails one by one it would be more efficient to post something publicly; so — for the benefit of the dozen or so people who want to port operating systems to run in EC2, and the curiosity of maybe a thousand more people who use EC2 but will never build AMIs themselves — here’s a rough guide to building EC2 images. Before we can talk about building images, there are some things you need: Your OS needs to run on x86 hardware. 64-bit (“amd64”, “x86-64”) is ideal, but I’ve managed to run 32-bit FreeBSD on “64-bit” EC2 instances so at least in some cases that’s not strictly necessary. You almost certainly want to have drivers for Xen block devices (for all of the pre-Nitro EC2 instances) or for NVMe disks (for the most recent EC2 instances). Theoretically you could make do without these since there’s some ATA emulation available for bootstrapping, but if you want to do any disk I/O after the kernel finishes booting you’ll want to have a disk driver. Similarly, you need support for the Xen network interface (older instances), Intel 10 GbE SR-IOV networking (some newer but pre-Nitro instances), or Amazon’s “ENA” network adapters (on Nitro instances), unless you plan on having instances which don’t communicate over the network. The ENA driver is probably the hardest thing to port, since as far as I know there’s no way to get your hands on the hardware directly, and it’s very difficult to do any debugging in EC2 without having a working network. Finally, the obvious: You need to have an AWS account, and appropriate API access keys. Building a disk image Building an AMI I wrote a simple tool for converting disk images into EC2 instances: bsdec2-image-upload. It uploads a disk image to Amazon S3; makes an API call to import that disk image into an EBS volume; creates a snapshot of that volume; then registers an EC2 AMI using that snapshot. To use bsdec2-image-upload, you’ll first need to create an S3 bucket for it to use as a staging area. You can call it anything you like, but I recommend that you Create it in a “nearby” region (for performance reasons), and Set an S3 “lifecycle policy” which deletes objects automatically after 1 day (since bsdec2-image-upload doesn’t clean up the S3 bucket, and those objects are useless once you’ve finished creating an AMI). Boot configuration Odds are that your instance started booting and got as far as the boot loader launching the kernel, but at some point after that things went sideways. Now we start the iterative process of building disk images, turning them into AMIs, launching said AMIs, and seeing where they break. Some things you’ll probably run into here: EC2 instances have two types of console available to them: A serial console and an VGA console. (Or rather, emulated serial and emulated VGA.) If you can have your kernel output go to both consoles, I recommend doing that. If you have to pick one, the serial console (which shows up as the “System Log” in EC2) is probably more useful than the VGA console (which shows up as “instance screenshot”) since it lets you see more than one screen of logs at once; but there’s a catch: Due to some bizarre breakage in EC2 — which I’ve been complaining about for ten years — the serial console is very “laggy”. If you find that you’re not getting any output, wait five minutes and try again. You may need to tell your kernel where to find the root filesystem. On FreeBSD we build our disk images using GPT labels, so we simply need to specify in /etc/fstab that the root filesystem is on /dev/gpt/rootfs; but if you can’t do this, you’ll probably need to have different AMIs for Nitro instances vs. non-Nitro instances since Xen block devices will typically show up with different device names from NVMe disks. On FreeBSD, I also needed to set the vfs.root.mountfrom kernel environment variable for a while; this also is no longer needed on FreeBSD but something similar may be needed on other systems. You’ll need to enable networking, using DHCP. On FreeBSD, this means placing ifconfigDEFAULT=“SYNCDHCP” into /etc/rc.conf; other systems will have other ways of specifying network parameters, and it may be necessary to specify a setting for the Xen network device, Intel SR-IOV network, and the Amazon ENA interface so that you’ll have the necessary configuration across all EC2 instance types. (On FreeBSD, ifconfigDEFAULT takes care of specifying the network settings which should apply for whatever network interface the kernel finds at boot time.) You’ll almost certainly want to turn on SSH, so that you can connect into newly launched instances and make use of them. Don’t worry about setting a password or creating a user to SSH into yet — we’ll take care of that later. EC2 configuration Now it’s time to make the AMI behave like an EC2 instance. To this end, I prepared a set of rc.d scripts for FreeBSD. Most importantly, they Print the SSH host keys to the console, so that you can veriy that they are correct when you first SSH in. (Remember, Verifying SSH host keys is more important than flossing every day.) Download the SSH public key you want to use for logging in, and create an account (by default, “ec2-user”) with that key set up for you. Fetch EC2 user-data and process it via configinit to allow you to configure the system as part of the process of launching it. If your OS has an rc system derived from NetBSD’s rc.d, you may be able to use these scripts without any changes by simply installing them and enabling them in /etc/rc.conf; otherwise you may need to write your own scripts using mine as a model. Firstboot scripts A feature I added to FreeBSD a few years ago is the concept of “firstboot” scripts: These startup scripts are only run the first time a system boots. The aforementioned configinit and SSH key fetching scripts are flagged this way — so if your OS doesn’t support the “firstboot” keyword on rc.d scripts you’ll need to hack around that — but EC2 instances also ship with other scripts set to run on the first boot: FreeBSD Update will fetch and install security and critical errata updates, and then reboot the system if necessary. The UFS filesystem on the “boot disk” will be automatically expanded to the full size of the disk — this makes it possible to specify a larger size of disk at EC2 instance launch time. Third-party packages will be automatically fetched and installed, according to a list in /etc/rc.conf. This is most useful if configinit is used to edit /etc/rc.conf, since it allows you to specify packages to install via the EC2 user-data. While none of these are strictly necessary, I find them to be extremely useful and highly recommend implementing similar functionality in your systems. Support my work! I hope you find this useful, or at very least interesting. Please consider supporting my work in this area; while I’m happy to contribute my time to supporting open source software, it would be nice if I had money coming in which I could use to cover incidental expenses (e.g., conference travel) so that I didn’t end up paying to contribute to FreeBSD. Digital Ocean https://do.co/bsdnow ###Traceability, by Vint Cerf A recent article from the August issue of the Communications of the ACM, for your contemplation: At a recent workshop on cybersecurity in the U.K., a primary topic of consideration was how to preserve the freedom and openness of the Internet while protecting against the harmful behaviors that have emerged in this global medium. That this is a significant challenge cannot be overstated. The bad behaviors range from social network bullying and misinformation to email spam, distributed denial of service attacks, direct cyberattacks against infrastructure, malware propagation, identity theft, and a host of other ills requiring a wide range of technical and legal considerations. That these harmful behaviors can and do cross international boundaries only makes it more difficult to fashion effective responses. In other columns, I have argued for better software development tools to reduce the common mistakes that lead to vulnerabilities that are exploited. Here, I want to focus on another aspect of response related to law enforcement and tracking down perpetrators. Of course, not all harms are (or perhaps are not yet) illegal, but discovering those who cause them may still be warranted. The recent adoption and implementation of the General Data Protection Regulation (GDPR) in the European Union creates an interesting tension because it highlights the importance and value of privacy while those who do direct or indirect harm must be tracked down and their identities discovered. In passing, I mention that cryptography has sometimes been blamed for protecting the identity or actions of criminals but it is also a tool for protecting privacy. Arguments have been made for “back doors” to cryptographic systems but I am of the opinion that such proposals carry extremely high risk to privacy and safety. It is not my intent to argue this question in this column. What is of interest to me is a concept to which I was introduced at the Ditchley workshop, specifically, differential traceability. The ability to trace bad actors to bring them to justice seems to me an important goal in a civilized society. The tension with privacy protection leads to the idea that only under appropriate conditions can privacy be violated. By way of example, consider license plates on cars. They are usually arbitrary identifiers and special authority is needed to match them with the car owners (unless, of course, they are vanity plates like mine: “Cerfsup”). This is an example of differential traceability; the police department has the authority to demand ownership information from the Department of Motor Vehicles that issues the license plates. Ordinary citizens do not have this authority. In the Internet environment there are a variety of identifiers associated with users (including corporate users). Domain names, IP addresses, email addresses, and public cryptography keys are examples among many others. Some of these identifiers are dynamic and thus ambiguous. For example, IP addresses are not always permanent and may change (for example, temporary IP addresses assigned at Wi-Fi hotspots) or may be ambiguous in the case of Network Address Translation. Information about the time of assignment and the party to whom an IP address was assigned may be needed to identify an individual user. There has been considerable debate and even a recent court case regarding requirements to register users in domain name WHOIS databases in the context of the adoption of GDPR. If we are to accomplish the simultaneous objectives of protecting privacy while apprehending those engaged in harmful or criminal behavior on the Internet, we must find some balance between conflicting but desirable outcomes. This suggests to me that the notion of traceability under (internationally?) agreed circumstances (that is, differential traceability) might be a fruitful concept to explore. In most societies today, it is accepted that we must be identifiable to appropriate authorities under certain conditions (consider border crossings, traffic violation stops as examples). While there are conditions under which apparent anonymity is desirable and even justifiable (whistle-blowing, for example) absolute anonymity is actually quite difficult to achieve (another point made at the Ditchley workshop) and might not be absolutely desirable given the misbehaviors apparent anonymity invites. I expect this is a controversial conclusion and I look forward to subsequent discussion. ###Remote Access Console using FreeBSD on an RPi3 Our friend, and FOSDEM Booth Neighbour, Jorge, has posted a tutorial on how he created a remote access console for his SmartOS server and other machines in his homelab Parts: Raspberry Pi 3 B+ NavoLabs micro POE Hat FT4232H based USB-to-RS232 (4x) adapter Official Raspberry Pi case (optional) Heat-sink kit (optional) USB-to-TTL adaptor (optional) Sandisk 16Gb microSD For the software I ended up using conserver. Below is a very brief tutorial on how to set everything up. I assume you have basic unix skills. Get an RPi3 image, make some minor modifications for RPi3+, and write it to the USB stick Configure FreeBSD on the RPi3 Load the ‘muge’ Ethernet Driver Load USB serial support Load the FTDI driver Enable SSHd and Conserver Configure Conserver Setup log rotation Start Conserver And you’re good to go A small bonus script I wrote to turn on the 2nd LED on the rPI once the system is booted, it will then blink the LED if someone is connected to any of the consoles. There is also a followup post with some additional tips: https://blackdot.be/2018/08/freebsd-uart-and-raspberry-pi-3-b/ ##Beastie Bits Annual Penguin Races Mscgen - Message Sequence Chart generator This patch makes FreeBSD boot 500 - 800ms faster, please test on your hardware FreeBSD’s arc4random() replaced with OpenBSD ChaCha20 implementation MeetBSD Devsummit open for registrations New Podcast interview with Michael W. Lucas Tarsnap ##Feedback/Questions We need more feedback emails. Please write to feedback@bsdnow.tv Additionally, we are considering a new segment to be added to the end of the show (to make it skippable), where we have a ~15 minute deep dive on a topic. Some initial ideas are on the Virtual Memory subsystem, the Scheduler, Capsicum, and GEOM. What topics would you like to get very detailed explanations of? Many of the explanations may have accompanying graphics, and not be very suitable for audio only listeners, that is why we are planning to put it at the very end of the episode. Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Episode 259: Long Live Unix | BSD Now 259
The strange birth and long life of Unix, FreeBSD jail with a single public IP, EuroBSDcon 2018 talks and schedule, OpenBSD on G4 iBook, PAM template user, ZFS file server, and reflections on one year of OpenBSD use. Picking the contest winner Vincent Bostjan Andrew Klaus-Hendrik Will Toby Johnny David manfrom Niclas Gary Eddy Bruce Lizz Jim Random number generator ##Headlines ###The Strange Birth and Long Life of Unix They say that when one door closes on you, another opens. People generally offer this bit of wisdom just to lend some solace after a misfortune. But sometimes it’s actually true. It certainly was for Ken Thompson and the late Dennis Ritchie, two of the greats of 20th-century information technology, when they created the Unix operating system, now considered one of the most inspiring and influential pieces of software ever written. A door had slammed shut for Thompson and Ritchie in March of 1969, when their employer, the American Telephone & Telegraph Co., withdrew from a collaborative project with the Massachusetts Institute of Technology and General Electric to create an interactive time-sharing system called Multics, which stood for “Multiplexed Information and Computing Service.” Time-sharing, a technique that lets multiple people use a single computer simultaneously, had been invented only a decade earlier. Multics was to combine time-sharing with other technological advances of the era, allowing users to phone a computer from remote terminals and then read e-mail, edit documents, run calculations, and so forth. It was to be a great leap forward from the way computers were mostly being used, with people tediously preparing and submitting batch jobs on punch cards to be run one by one. Over five years, AT&T invested millions in the Multics project, purchasing a GE-645 mainframe computer and dedicating to the effort many of the top researchers at the company’s renowned Bell Telephone Laboratories—including Thompson and Ritchie, Joseph F. Ossanna, Stuart Feldman, M. Douglas McIlroy, and the late Robert Morris. But the new system was too ambitious, and it fell troublingly behind schedule. In the end, AT&T’s corporate leaders decided to pull the plug. After AT&T’s departure from the Multics project, managers at Bell Labs, in Murray Hill, N.J., became reluctant to allow any further work on computer operating systems, leaving some researchers there very frustrated. Although Multics hadn’t met many of its objectives, it had, as Ritchie later recalled, provided them with a “convenient interactive computing service, a good environment in which to do programming, [and] a system around which a fellowship could form.” Suddenly, it was gone. With heavy hearts, the researchers returned to using their old batch system. At such an inauspicious moment, with management dead set against the idea, it surely would have seemed foolhardy to continue designing computer operating systems. But that’s exactly what Thompson, Ritchie, and many of their Bell Labs colleagues did. Now, some 40 years later, we should be thankful that these programmers ignored their bosses and continued their labor of love, which gave the world Unix, one of the greatest computer operating systems of all time. The rogue project began in earnest when Thompson, Ritchie, and a third Bell Labs colleague, Rudd Canaday, began to sketch out on paper the design for a file system. Thompson then wrote the basics of a new operating system for the lab’s GE-645 mainframe. But with the Multics project ended, so too was the need for the GE-645. Thompson realized that any further programming he did on it was likely to go nowhere, so he dropped the effort. Thompson had passed some of his time after the demise of Multics writing a computer game called Space Travel, which simulated all the major bodies in the solar system along with a spaceship that could fly around them. Written for the GE-645, Space Travel was clunky to play—and expensive: roughly US $75 a game for the CPU time. Hunting around, Thompson came across a dusty PDP-7, a minicomputer built by Digital Equipment Corp. that some of his Bell Labs colleagues had purchased earlier for a circuit-analysis project. Thompson rewrote Space Travel to run on it. And with that little programming exercise, a second door cracked ajar. It was to swing wide open during the summer of 1969 when Thompson’s wife, Bonnie, spent a month visiting his parents to show off their newborn son. Thompson took advantage of his temporary bachelor existence to write a good chunk of what would become the Unix operating system for the discarded PDP‑7. The name Unix stems from a joke one of Thompson’s colleagues made: Because the new operating system supported only one user (Thompson), he saw it as an emasculated version of Multics and dubbed it “Un-multiplexed Information and Computing Service,” or Unics. The name later morphed into Unix. Initially, Thompson used the GE-645 to compose and compile the software, which he then downloaded to the PDP‑7. But he soon weaned himself from the mainframe, and by the end of 1969 he was able to write operating-system code on the PDP-7 itself. That was a step in the right direction. But Thompson and the others helping him knew that the PDP‑7, which was already obsolete, would not be able to sustain their skunkworks for long. They also knew that the lab’s management wasn’t about to allow any more research on operating systems. So Thompson and Ritchie got creative. They formulated a proposal to their bosses to buy one of DEC’s newer minicomputers, a PDP-11, but couched the request in especially palatable terms. They said they were aiming to create tools for editing and formatting text, what you might call a word-processing system today. The fact that they would also have to write an operating system for the new machine to support the editor and text formatter was almost a footnote. Management took the bait, and an order for a PDP-11 was placed in May 1970. The machine itself arrived soon after, although the disk drives for it took more than six months to appear. During the interim, Thompson, Ritchie, and others continued to develop Unix on the PDP-7. After the PDP-11’s disks were installed, the researchers moved their increasingly complex operating system over to the new machine. Next they brought over the roff text formatter written by Ossanna and derived from the runoff program, which had been used in an earlier time-sharing system. Unix was put to its first real-world test within Bell Labs when three typists from AT&T’s patents department began using it to write, edit, and format patent applications. It was a hit. The patent department adopted the system wholeheartedly, which gave the researchers enough credibility to convince management to purchase another machine—a newer and more powerful PDP-11 model—allowing their stealth work on Unix to continue. During its earliest days, Unix evolved constantly, so the idea of issuing named versions or releases seemed inappropriate. But the researchers did issue new editions of the programmer’s manual periodically, and the early Unix systems were named after each such edition. The first edition of the manual was completed in November 1971. So what did the first edition of Unix offer that made it so great? For one thing, the system provided a hierarchical file system, which allowed something we all now take for granted: Files could be placed in directories—or equivalently, folders—that in turn could be put within other directories. Each file could contain no more than 64 kilobytes, and its name could be no more than six characters long. These restrictions seem awkwardly limiting now, but at the time they appeared perfectly adequate. Although Unix was ostensibly created for word processing, the only editor available in 1971 was the line-oriented ed. Today, ed is still the only editor guaranteed to be present on all Unix systems. Apart from the text-processing and general system applications, the first edition of Unix included games such as blackjack, chess, and tic-tac-toe. For the system administrator, there were tools to dump and restore disk images to magnetic tape, to read and write paper tapes, and to create, check, mount, and unmount removable disk packs. Most important, the system offered an interactive environment that by this time allowed time-sharing, so several people could use a single machine at once. Various programming languages were available to them, including BASIC, Fortran, the scripting of Unix commands, assembly language, and B. The last of these, a descendant of a BCPL (Basic Combined Programming Language), ultimately evolved into the immensely popular C language, which Ritchie created while also working on Unix. The first edition of Unix let programmers call 34 different low-level routines built into the operating system. It’s a testament to the system’s enduring nature that nearly all of these system calls are still available—and still heavily used—on modern Unix and Linux systems four decades on. For its time, first-edition Unix provided a remarkably powerful environment for software development. Yet it contained just 4200 lines of code at its heart and occupied a measly 16 KB of main memory when it ran. Unix’s great influence can be traced in part to its elegant design, simplicity, portability, and serendipitous timing. But perhaps even more important was the devoted user community that soon grew up around it. And that came about only by an accident of its unique history. The story goes like this: For years Unix remained nothing more than a Bell Labs research project, but by 1973 its authors felt the system was mature enough for them to present a paper on its design and implementation at a symposium of the Association for Computing Machinery. That paper was published in 1974 in the Communications of the ACM. Its appearance brought a flurry of requests for copies of the software. This put AT&T in a bind. In 1956, AT&T had agreed to a U.S government consent decree that prevented the company from selling products not directly related to telephones and telecommunications, in return for its legal monopoly status in running the country’s long-distance phone service. So Unix could not be sold as a product. Instead, AT&T released the Unix source code under license to anyone who asked, charging only a nominal fee. The critical wrinkle here was that the consent decree prevented AT&T from supporting Unix. Indeed, for many years Bell Labs researchers proudly displayed their Unix policy at conferences with a slide that read, “No advertising, no support, no bug fixes, payment in advance.” With no other channels of support available to them, early Unix adopters banded together for mutual assistance, forming a loose network of user groups all over the world. They had the source code, which helped. And they didn’t view Unix as a standard software product, because nobody seemed to be looking after it. So these early Unix users themselves set about fixing bugs, writing new tools, and generally improving the system as they saw fit. The Usenix user group acted as a clearinghouse for the exchange of Unix software in the United States. People could send in magnetic tapes with new software or fixes to the system and get back tapes with the software and fixes that Usenix had received from others. In Australia, the University of New South Wales and the University of Sydney produced a more robust version of Unix, the Australian Unix Share Accounting Method, which could cope with larger numbers of concurrent users and offered better performance. By the mid-1970s, the environment of sharing that had sprung up around Unix resembled the open-source movement so prevalent today. Users far and wide were enthusiastically enhancing the system, and many of their improvements were being fed back to Bell Labs for incorporation in future releases. But as Unix became more popular, AT&T’s lawyers began looking harder at what various licensees were doing with their systems. One person who caught their eye was John Lions, a computer scientist then teaching at the University of New South Wales, in Australia. In 1977, he published what was probably the most famous computing book of the time, A Commentary on the Unix Operating System, which contained an annotated listing of the central source code for Unix. Unix’s licensing conditions allowed for the exchange of source code, and initially, Lions’s book was sold to licensees. But by 1979, AT&T’s lawyers had clamped down on the book’s distribution and use in academic classes. The antiauthoritarian Unix community reacted as you might expect, and samizdat copies of the book spread like wildfire. Many of us have nearly unreadable nth-generation photocopies of the original book. End runs around AT&T’s lawyers indeed became the norm—even at Bell Labs. For example, between the release of the sixth edition of Unix in 1975 and the seventh edition in 1979, Thompson collected dozens of important bug fixes to the system, coming both from within and outside of Bell Labs. He wanted these to filter out to the existing Unix user base, but the company’s lawyers felt that this would constitute a form of support and balked at their release. Nevertheless, those bug fixes soon became widely distributed through unofficial channels. For instance, Lou Katz, the founding president of Usenix, received a phone call one day telling him that if he went down to a certain spot on Mountain Avenue (where Bell Labs was located) at 2 p.m., he would find something of interest. Sure enough, Katz found a magnetic tape with the bug fixes, which were rapidly in the hands of countless users. By the end of the 1970s, Unix, which had started a decade earlier as a reaction against the loss of a comfortable programming environment, was growing like a weed throughout academia and the IT industry. Unix would flower in the early 1980s before reaching the height of its popularity in the early 1990s. For many reasons, Unix has since given way to other commercial and noncommercial systems. But its legacy, that of an elegant, well-designed, comfortable environment for software development, lives on. In recognition of their accomplishment, Thompson and Ritchie were given the Japan Prize earlier this year, adding to a collection of honors that includes the United States’ National Medal of Technology and Innovation and the Association of Computing Machinery’s Turing Award. Many other, often very personal, tributes to Ritchie and his enormous influence on computing were widely shared after his death this past October. Unix is indeed one of the most influential operating systems ever invented. Its direct descendants now number in the hundreds. On one side of the family tree are various versions of Unix proper, which began to be commercialized in the 1980s after the Bell System monopoly was broken up, freeing AT&T from the stipulations of the 1956 consent decree. On the other side are various Unix-like operating systems derived from the version of Unix developed at the University of California, Berkeley, including the one Apple uses today on its computers, OS X. I say “Unix-like” because the developers of the Berkeley Software Distribution (BSD) Unix on which these systems were based worked hard to remove all the original AT&T code so that their software and its descendants would be freely distributable. The effectiveness of those efforts were, however, called into question when the AT&T subsidiary Unix System Laboratories filed suit against Berkeley Software Design and the Regents of the University of California in 1992 over intellectual property rights to this software. The university in turn filed a counterclaim against AT&T for breaches to the license it provided AT&T for the use of code developed at Berkeley. The ensuing legal quagmire slowed the development of free Unix-like clones, including 386BSD, which was designed for the Intel 386 chip, the CPU then found in many IBM PCs. Had this operating system been available at the time, Linus Torvalds says he probably wouldn’t have created Linux, an open-source Unix-like operating system he developed from scratch for PCs in the early 1990s. Linux has carried the Unix baton forward into the 21st century, powering a wide range of digital gadgets including wireless routers, televisions, desktop PCs, and Android smartphones. It even runs some supercomputers. Although AT&T quickly settled its legal disputes with Berkeley Software Design and the University of California, legal wrangling over intellectual property claims to various parts of Unix and Linux have continued over the years, often involving byzantine corporate relations. By 2004, no fewer than five major lawsuits had been filed. Just this past August, a software company called the TSG Group (formerly known as the SCO Group), lost a bid in court to claim ownership of Unix copyrights that Novell had acquired when it purchased the Unix System Laboratories from AT&T in 1993. As a programmer and Unix historian, I can’t help but find all this legal sparring a bit sad. From the very start, the authors and users of Unix worked as best they could to build and share, even if that meant defying authority. That outpouring of selflessness stands in sharp contrast to the greed that has driven subsequent legal battles over the ownership of Unix. The world of computer hardware and software moves forward startlingly fast. For IT professionals, the rapid pace of change is typically a wonderful thing. But it makes us susceptible to the loss of our own history, including important lessons from the past. To address this issue in a small way, in 1995 I started a mailing list of old-time Unix aficionados. That effort morphed into the Unix Heritage Society. Our goal is not only to save the history of Unix but also to collect and curate these old systems and, where possible, bring them back to life. With help from many talented members of this society, I was able to restore much of the old Unix software to working order, including Ritchie’s first C compiler from 1972 and the first Unix system to be written in C, dating from 1973. One holy grail that eluded us for a long time was the first edition of Unix in any form, electronic or otherwise. Then, in 2006, Al Kossow from the Computer History Museum, in Mountain View, Calif., unearthed a printed study of Unix dated 1972, which not only covered the internal workings of Unix but also included a complete assembly listing of the kernel, the main component of this operating system. This was an amazing find—like discovering an old Ford Model T collecting dust in a corner of a barn. But we didn’t just want to admire the chrome work from afar. We wanted to see the thing run again. In 2008, Tim Newsham, an independent programmer in Hawaii, and I assembled a team of like-minded Unix enthusiasts and set out to bring this ancient system back from the dead. The work was technically arduous and often frustrating, but in the end, we had a copy of the first edition of Unix running on an emulated PDP-11/20. We sent out messages announcing our success to all those we thought would be interested. Thompson, always succinct, simply replied, “Amazing.” Indeed, his brainchild was amazing, and I’ve been happy to do what I can to make it, and the story behind it, better known. Digital Ocean http://do.co/bsdnow ###FreeBSD jails with a single public IP address Jails in FreeBSD provide a simple yet flexible way to set up a proper server layout. In the most setups the actual server only acts as the host system for the jails while the applications themselves run within those independent containers. Traditionally every jail has it’s own IP for the user to be able to address the individual services. But if you’re still using IPv4 this might get you in trouble as the most hosters don’t offer more than one single public IP address per server. Create the internal network In this case NAT (“Network Address Translation”) is a good way to expose services in different jails using the same IP address. First, let’s create an internal network (“NAT network”) at 192.168.0.0/24. You could generally use any private IPv4 address space as specified in RFC 1918. Here’s an overview: https://en.wikipedia.org/wiki/Privatenetwork. Using pf, FreeBSD’s firewall, we will map requests on different ports of the same public IP address to our individual jails as well as provide network access to the jails themselves. First let’s check which network devices are available. In my case there’s em0 which provides connectivity to the internet and lo0, the local loopback device. options=209b<RXCSUM,TXCSUM,VLANMTU,VLANHWTAGGING,VLANHWCSUM,WOLMAGIC> [...] inet 172.31.1.100 netmask 0xffffff00 broadcast 172.31.1.255 nd6 options=23<PERFORMNUD,ACCEPTRTADV,AUTO_LINKLOCAL> media: Ethernet autoselect (1000baseT <full-duplex>) status: active lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384 options=600003<RXCSUM,TXCSUM,RXCSUMIPV6,TXCSUMIPV6> inet6 ::1 prefixlen 128 inet6 fe80::1%lo0 prefixlen 64 scopeid 0x2 inet 127.0.0.1 netmask 0xff000000 nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>``` > For our internal network, we create a cloned loopback device called lo1. Therefore we need to customize the /etc/rc.conf file, adding the following two lines: cloned_interfaces="lo1" ipv4_addrs_lo1="192.168.0.1-9/29" > This defines a /29 network, offering IP addresses for a maximum of 6 jails: ipcalc 192.168.0.1/29 Address: 192.168.0.1 11000000.10101000.00000000.00000 001 Netmask: 255.255.255.248 = 29 11111111.11111111.11111111.11111 000 Wildcard: 0.0.0.7 00000000.00000000.00000000.00000 111 => Network: 192.168.0.0/29 11000000.10101000.00000000.00000 000 HostMin: 192.168.0.1 11000000.10101000.00000000.00000 001 HostMax: 192.168.0.6 11000000.10101000.00000000.00000 110 Broadcast: 192.168.0.7 11000000.10101000.00000000.00000 111 Hosts/Net: 6 Class C, Private Internet > Then we need to restart the network. Please be aware of currently active SSH sessions as they might be dropped during restart. It’s a good moment to ensure you have KVM access to that server ;-) service netif restart > After reconnecting, our newly created loopback device is active: lo1: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384 options=600003<RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6> inet 192.168.0.1 netmask 0xfffffff8 inet 192.168.0.2 netmask 0xffffffff inet 192.168.0.3 netmask 0xffffffff inet 192.168.0.4 netmask 0xffffffff inet 192.168.0.5 netmask 0xffffffff inet 192.168.0.6 netmask 0xffffffff inet 192.168.0.7 netmask 0xffffffff inet 192.168.0.8 netmask 0xffffffff inet 192.168.0.9 netmask 0xffffffff nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL> Setting up > pf part of the FreeBSD base system, so we only have to configure and enable it. By this moment you should already have a clue of which services you want to expose. If this is not the case, just fix that file later on. In my example configuration, I have a jail running a webserver and another jail running a mailserver: Public IP address IP_PUB="1.2.3.4" Packet normalization scrub in all Allow outbound connections from within the jails nat on em0 from lo1:network to any -> (em0) webserver jail at 192.168.0.2 rdr on em0 proto tcp from any to $IP_PUB port 443 -> 192.168.0.2 just an example in case you want to redirect to another port within your jail rdr on em0 proto tcp from any to $IP_PUB port 80 -> 192.168.0.2 port 8080 mailserver jail at 192.168.0.3 rdr on em0 proto tcp from any to $IP_PUB port 25 -> 192.168.0.3 rdr on em0 proto tcp from any to $IP_PUB port 587 -> 192.168.0.3 rdr on em0 proto tcp from any to $IP_PUB port 143 -> 192.168.0.3 rdr on em0 proto tcp from any to $IP_PUB port 993 -> 192.168.0.3 > Now just enable pf like this (which is the equivalent of adding pf_enable=YES to /etc/rc.conf): sysrc pf_enable="YES" > and start it: service pf start Install ezjail > Ezjail is a collection of scripts by erdgeist that allow you to easily manage your jails. pkg install ezjail > As an alternative, you could install ezjail from the ports tree. Now we need to set up the basejail which contains the shared base system for our jails. In fact, every jail that you create get’s will use that basejail to symlink directories related to the base system like /bin and /sbin. This can be accomplished by running ezjail-admin install > In the next step, we’ll copy the /etc/resolv.conf file from our host to the newjail, which is the template for newly created jails (the parts that are not provided by basejail), to ensure that domain resolution will work properly within our jails later on: cp /etc/resolv.conf /usr/jails/newjail/etc/ > Last but not least, we enable ezjail and start it: sysrc ezjail_enable="YES" service ezjail start Create a jail > Creating a jail is as easy as it could probably be: ezjail-admin create webserver 192.168.0.2 ezjail-admin start webserver > Now you can access your jail using: ezjail-admin console webserver > Each jail contains a vanilla FreeBSD installation. Deploy services > Now you can spin up as many jails as you want to set up your services like web, mail or file shares. You should take care not to enable sshd within your jails, because that would cause problems with the service’s IP bindings. But this is not a problem, just SSH to the host and enter your jail using ezjail-admin console. EuroBSDcon 2018 Talks & Schedule (https://2018.eurobsdcon.org/talks-schedule/) News Roundup OpenBSD on an iBook G4 (https://bobstechsite.com/openbsd-on-an-ibook-g4/) > I've mentioned on social media and on the BTS podcast a few times that I wanted to try installing OpenBSD onto an old "snow white" iBook G4 I acquired last summer to see if I could make it a useful machine again in the year 2018. This particular eBay purchase came with a 14" 1024x768 TFT screen, 1.07GHz PowerPC G4 processor, 1.5GB RAM, 100GB of HDD space and an ATI Radeon 9200 graphics card with 32 MB of SDRAM. The optical drive, ethernet port, battery & USB slots are also fully-functional. The only thing that doesn't work is the CMOS battery, but that's not unexpected for a device that was originally released in 2004. Initial experiments > This iBook originally arrived at my door running Apple Mac OSX Leopard and came with the original install disk, the iLife & iWork suites for 2008, various instruction manuals, a working power cable and a spare keyboard. As you'll see in the pictures I took for this post the characters on the buttons have started to wear away from 14 years of intensive use, but the replacement needs a very good clean before I decide to swap it in! > After spending some time exploring the last version of OSX to support the IBM PowerPC processor architecture I tried to see if the hardware was capable of modern computing with Linux. Something I knew ahead of trying this was that the WiFi adapter was unlikely to work because it's a highly proprietary component designed by Apple to work specifically with OSX and nothing else, but I figured I could probably use a wireless USB dongle later to get around this limitation. > Unfortunately I found that no recent versions of mainstream Linux distributions would boot off this machine. Debian has dropped support 32-bit PowerPC architectures and the PowerPC variants of Ubuntu 16.04 LTS (vanilla, MATE and Lubuntu) wouldn't even boot the installer! The only distribution I could reliably install on the hardware was Lubuntu 14.04 LTS. > Unfortunately I'm not the biggest fan of the LXDE desktop for regular work and a lot of ported applications were old and broken because it clearly wasn't being maintained by people that use the hardware anymore. Ubuntu 14.04 is also approaching the end of its support life in early 2019, so this limited solution also has a limited shelf-life. Over to BSD > I discussed this problem with a few people on Mastodon and it was pointed out to me that OSX is built on the Darwin kernel, which happens to be a variant of BSD. NetBSD and OpenBSD fans in particular convinced me that their communities still saw the value of supporting these old pieces of kit and that I should give BSD a try. > So yesterday evening I finally downloaded the "macppc" version of OpenBSD 6.3 with no idea what to expect. I hoped for the best but feared the worst because my last experience with this operating system was trying out PC-BSD in 2008 and discovering with disappointment that it didn't support any of the hardware on my Toshiba laptop. > When I initially booted OpenBSD I was a little surprised to find the login screen provided no visual feedback when I typed in my password, but I can understand the security reasons for doing that. The initial desktop environment that was loaded was very basic. All I could see was a console output window, a terminal and a desktop switcher in the X11 environment the system had loaded. > After a little Googling I found this blog post had some fantastic instructions to follow for the post-installation steps: https://sohcahtoa.org.uk/openbsd.html. I did have to adjust them slightly though because my iBook only has 1.5GB RAM and not every package that page suggests is available on macppc by default. You can see a full list here: https://ftp.openbsd.org/pub/OpenBSD/6.3/packages/powerpc/. Final thoughts > I was really impressed with the performance of OpenBSD's "macppc" port. It boots much faster than OSX Leopard on the same hardware and unlike Lubuntu 14.04 it doesn't randomly hang for no reason or crash if you launch something demanding like the GIMP. > I was pleased to see that the command line tools I'm used to using on Linux have been ported across too. OpenBSD also had no issues with me performing basic desktop tasks on XFCE like browsing the web with NetSurf, playing audio files with VLC and editing images with the GIMP. Limited gaming is also theoretically possible if you're willing to build them (or an emulator) from source with SDL support. > If I wanted to use this system for heavy duty work then I'd probably be inclined to run key applications like LibreOffice on a Raspberry Pi and then connect my iBook G4 to those using VNC or an SSH connection with X11 forwarding. BSD is UNIX after all, so using my ancient laptop as a dumb terminal should work reasonably well. > In summary I was impressed with OpenBSD and its ability to breathe new life into this old Apple Mac. I'm genuinely excited about the idea of trying BSD with other devices on my network such as an old Asus Eee PC 900 netbook and at least one of the many Raspberry Pi devices I use. Whether I go the whole hog and replace Fedora on my main production laptop though remains to be seen! The template user with PAM and login(1) (http://oshogbo.vexillium.org/blog/48) > When you build a new service (or an appliance) you need your users to be able to configure it from the command line. To accomplish this you can create system accounts for all registered users in your service and assign them a special login shell which provides such limited functionality. This can be painful if you have a dynamic user database. > Another challenge is authentication via remote services such as RADIUS. How can we implement services when we authenticate through it and log into it as a different user? Furthermore, imagine a scenario when RADIUS decides on which account we have the right to access by sending an additional attribute. > To address these two problems we can use a "template" user. Any of the PAM modules can set the value of the PAM_USER item. The value of this item will be used to determine which account we want to login. Only the "template" user must exist on the local password database, but the credential check can be omitted by the module. > This functionality exists in the login(1) used by FreeBSD, HardenedBSD, DragonFlyBSD and illumos. The functionality doesn't exist in the login(1) used in NetBSD, and OpenBSD doesn't support PAM modules at all. In addition what is also noteworthy is that such functionality was also in the OpenSSH but they decided to remove it and call it a security vulnerability (CVE 2015-6563). I can see how some people may have seen it that way, that’s why I recommend reading this article from an OpenPAM author and a FreeBSD security officer at the time. > Knowing the background let's take a look at an example. ```PAMEXTERN int pamsmauthenticate(pamhandlet *pamh, int flags _unused, int argc _unused, const char *argv[] _unused) { const char *user, *password; int err; err = pam_get_user(pamh, &user, NULL); if (err != PAM_SUCCESS) return (err); err = pam_get_authtok(pamh, PAM_AUTHTOK, &password, NULL); if (err == PAM_CONV_ERR) return (err); if (err != PAM_SUCCESS) return (PAM_AUTH_ERR); err = authenticate(user, password); if (err != PAM_SUCCESS) { return (err); } return (pam_set_item(pamh, PAM_USER, "template")); } In the listing above we have an example of a PAM module. The pamgetuser(3) provides a username. The pamgetauthtok(3) shows us a secret given by the user. Both functions allow us to give an optional prompt which should be shown to the user. The authenticate function is our crafted function which authenticates the user. In our first scenario we wanted to keep all users in an external database. If authentication is successful we then switch to a template user which has a shell set up for a script allowing us to configure the machine. In our second scenario the authenticate function authenticates the user in RADIUS. Another step is to add our PAM module to the /etc/pam.d/system or to the /etc/pam.d/login configuration: auth sufficient pamtemplate.so nowarn allowlocal Unfortunately the description of all these options goes beyond this article - if you would like to know more about it you can find them in the PAM manual. The last thing we need to do is to add our template user to the system which you can do by the adduser(8) command or just simply modifying the /etc/master.passwd file and use pwdmkdb(8) program: $ tail -n /etc/master.passwd template::1000:1000::0:0:User &:/:/usr/local/bin/templatesh $ sudo pwdmkdb /etc/master.passwd As you can see,the template user can be locked and we still can use it in our PAM module (the * character after login). I would like to thank Dag-Erling Smørgrav for pointing this functionality out to me when I was looking for it some time ago. iXsystems iXsystems @ VMWorld ###ZFS file server What is the need? At work, we run a compute cluster that uses an Isilon cluster as primary NAS storage. Excluding snapshots, we have about 200TB of research data, some of them in compressed formats, and others not. We needed an offsite backup file server that would constantly mirror our primary NAS and serve as a quick recovery source in case of a data loss in the the primary NAS. This offsite file server would be passive - will never face the wrath of the primary cluster workload. In addition to the role of a passive backup server, this solution would take on some passive report generation workloads as an ideal way of offloading some work from the primary NAS. The passive work is read-only. The backup server would keep snapshots in a best effort basis dating back to 10 years. However, this data on this backup server would be archived to tapes periodically. A simple guidance of priorities: Data integrity > Cost of solution > Storage capacity > Performance. Why not enterprise NAS? NetApp FAS or EMC Isilon or the like? We decided that enterprise grade NAS like NetAPP FAS or EMC Isilon are prohibitively expensive and an overkill for our needs. An open source & cheaper alternative to enterprise grade filesystem with the level of durability we expect turned up to be ZFS. We’re already spoilt from using snapshots by a clever Copy-on-Write Filesystem(WAFL) by NetApp. ZFS providing snapshots in almost identical way was a big influence in the choice. This is also why we did not consider just a CentOS box with the default XFS filesystem. FreeBSD vs Debian for ZFS This is a backup server, a long-term solution. Stability and reliability are key requirements. ZFS on Linux may be popular at this time, but there is a lot of churn around its development, which means there is a higher probability of bugs like this to occur. We’re not looking for cutting edge features here. Perhaps, Linux would be considered in the future. FreeBSD + ZFS We already utilize FreeBSD and OpenBSD for infrastructure services and we have nothing but praises for the stability that the BSDs have provided us. We’d gladly use FreeBSD and OpenBSD wherever possible. Okay, ZFS, but why not FreeNAS? IMHO, FreeNAS provides a integrated GUI management tool over FreeBSD for a novice user to setup and configure FreeBSD, ZFS, Jails and many other features. But, this user facing abstraction adds an extra layer of complexity to maintain that is just not worth it in simpler use cases like ours. For someone that appreciates the commandline interface, and understands FreeBSD enough to administer it, plain FreeBSD + ZFS is simpler and more robust than FreeNAS. Specifications Lenovo SR630 Rackserver 2 X Intel Xeon silver 4110 CPUs 768 GB of DDR4 ECC 2666 MHz RAM 4 port SAS card configured in passthrough mode(JBOD) Intel network card with 10 Gb SFP+ ports 128GB M.2 SSD for use as boot drive 2 X HGST 4U60 JBOD 120(2 X 60) X 10TB SAS disks ###Reflection on one-year usage of OpenBSD I have used OpenBSD for more than one year, and it is time to give a summary of the experience: (1) What do I get from OpenBSD? a) A good UNIX tutorial. When I am curious about some UNIXcommands’ implementation, I will refer to OpenBSD source code, and I actually gain something every time. E.g., refresh socket programming skills from nc; know how to process file efficiently from cat. b) A better test bed. Although my work focus on developing programs on Linux, I will try to compile and run applications on OpenBSD if it is possible. One reason is OpenBSD usually gives more helpful warnings. E.g., hint like this: ...... warning: sprintf() is often misused, please use snprintf() ...... Or you can refer this post which I wrote before. The other is sometimes program run well on Linux may crash on OpenBSD, and OpenBSD can help you find hidden bugs. c) Some handy tools. E.g. I find tcpbench is useful, so I ported it into Linux for my own usage (project is here). (2) What I give back to OpenBSD? a) Patches. Although most of them are trivial modifications, they are still my contributions. b) Write blog posts to share experience about using OpenBSD. c) Develop programs for OpenBSD/BSD: lscpu and free. d) Porting programs into OpenBSD: E.g., I find google/benchmark is a nifty tool, but lacks OpenBSD support, I submitted PR and it is accepted. So you can use google/benchmark on OpenBSD now. Generally speaking, the time invested on OpenBSD is rewarding. If you are still hesitating, why not give a shot? ##Beastie Bits BSD Users Stockholm Meetup BSDCan 2018 Playlist OPNsense 18.7 released Testing TrueOS (FreeBSD derivative) on real hardware ThinkPad T410 Kernel Hacker Wanted! Replace a pair of 8-bit writes to VGA memory with a single 16-bit write Reduce taskq and context-switch cost of zio pipe Proposed FreeBSD Memory Management change, expected to improve ZFS ARC interactions Tarsnap ##Feedback/Questions Anian_Z - Question Robert - Pool question Lain - Congratulations Thomas - L2arc Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Episode 258: OS Foundations | BSD Now 258
FreeBSD Foundation July Newsletter, a bunch of BSDCan trip reports, HardenedBSD Foundation status, FreeBSD and OSPFd, ZFS disk structure overview, and more Spectre mitigations in OpenBSD. ##Headlines FreeBSD Foundation Update, July 2018 MESSAGE FROM THE EXECUTIVE DIRECTOR We’re in the middle of summer here, in Boulder, CO. While the days are typically hot, they can also be quite unpredictable. Thanks to the Rocky Mountains, waking up to 50-degree (~10 C) foggy weather is not surprising. In spite of the unpredictable weather, many of us took some vacation this month. Whether it was extending the Fourth of July celebration, spending time with family, or relaxing and enjoying the summer weather, we appreciated our time off, while still managing to accomplish a lot! In this newsletter, Glen Barber enlightens us about the upcoming 12.0 release. I gave a recap of OSCON, that Ed Maste and I attended, and Mark Johnston explains the work on his improved microcode loading project, that we are funding. Finally, Anne Dickison gives us a rundown on upcoming events and information on submitting a talk for MeetBSD. Your support helps us continue this work. Please consider making a donation today. We can’t do it without you. Happy reading!! June 2018 Development Projects Update Fundraising Update: Supporting the Project July 2018 Release Engineering Update OSCON 2018 Recap Submit Your Work: MeetBSD 2018 FreeBSD Discount for 2018 SNIA Developer Conference EuroBSDcon 2018 Travel Grant Application Deadline: August 2 iXsystems ###BSDCan Trip Reports BSDCan 2018 Trip Report: Constantin Stan BSDCan 2018 Trip Report: Danilo G. Baio BSDCan 2018 Trip Report: Rodrigo Osorio BSDCan 2018 Trip Report: Dhananjay Balan BSDCan 2018 Trip Report: Kyle Evans ##News Roundup FreeBSD and OSPFd With FreeBSD jails deployed around the world, static routing was getting a bit out of hand. Plus, when I needed to move a jail from one data center to another, I would have to update routing tables across multiple sites. Not ideal. Enter dynamic routing… OSPF (open shortest path first) is an internal dynamic routing protocol that provides the autonomy that I needed and it’s fairly easy to setup. This article does not cover configuration of VPN links, ZFS, or Freebsd jails, however it’s recommended that you use seperate ZFS datasets per jail so that migration between hosts can be done with zfs send & receive. In this scenario, we have five FreeBSD servers in two different data centers. Each physical server runs anywhere between three to ten jails. When jails are deployed, they are assigned a /32 IP on lo2. From here, pf handles inbound port forwarding and outbound NAT. Links between each server are provided by OpenVPN TAP interfaces. (I used TAP to pass layer 2 traffic. I seem to remember that I needed TAP interfaces due to needing GRE tunnels on top of TUN interfaces to get OSPF to communicate. I’ve heard TAP is slower than TUN so I may revisit this.) In this example, we will use 172.16.2.0/24 as the range for OpenVPN P2P links and 172.16.3.0/24 as the range of IPs available for assignment to each jail. Previously, when deploying a jail, I assigned IPs based on the following groups: Server 1: 172.16.3.0/28 Server 2: 172.16.3.16/28 Server 3: 172.16.3.32/28 Server 4: 172.16.3.48/28 Server 5: 172.16.3.64/28 When statically routing, this made routing tables a bit smaller and easier to manage. However, when I needed to migrate a jail to a new host, I had to add a new /32 to all routing tables. Now, with OSPF, this is no longer an issue, nor is it required. To get started, first we install the Quagga package. The two configuration files needed to get OSPFv2 running are /usr/local/etc/quagga/zebra.conf and /usr/local/etc/quagga/ospfd.conf. Starting with zebra.conf, we’ll define the hostname and a management password. Second, we will populate the ospfd.conf file. To break this down: service advanced-vty allows you to skip the en or enable command. Since I’m the only one who uses this service, it’s one less command to type. ip ospf authentication message-digest and ip ospf message-diget-key… ignores non-authenticated OSPF communication. This is useful when communicating over the WAN and to prevent a replay attack. Since I’m using a VPN to communicate, I could exclude these. passive-interface default turns off the active communication of OSPF messages on all interfaces except for the interfaces listed as no passive-interface [interface name]. Since my ospf communication needs to leverage the VPNs, this prevents the servers from trying to send ospf data out the WAN interface (a firewall would work too). network 172.16.2.0/23 area 0.0.0.0 lists a supernet of both 172.16.2.0/24 and 172.16.3.0/24. This ensures routes for the jails are advertised along with the P2P links used by OpenVPN. The OpenVPN links are not required but can provide another IP to access your server if one of the links goes down. (See the suggested tasks below). At this point, we can enable the services in rc.conf.local and start them. We bind the management interface to 127.0.0.1 so that it’s only accessable to local telnet sessions. If you want to access this service remotely, you can bind to a remotely accessable IP. Remember telnet is not secure. If you need remote access, use a VPN. To manage the services, you can telnet to your host’s localhost address. Use 2604 for the ospf service. Remember, this is accessible by non-root users so set a good password. ###A broad overview of how ZFS is structured on disk When I wrote yesterday’s entry, it became clear that I didn’t understand as much about how ZFS is structured on disk (and that this matters, since I thought that ZFS copy on write updates updated a lot more than they do). So today I want to write down my new broad understanding of how this works. (All of this can be dug out of the old, draft ZFS on-disk format specification, but that spec is written in a very detailed way and things aren’t always immediately clear from it.) Almost everything in ZFS is in DMU object. All objects are defined by a dnode, and object dnodes are almost always grouped together in an object set. Object sets are themselves DMU objects; they store dnodes as basically a giant array in a ‘file’, which uses data blocks and indirect blocks and so on, just like anything else. Within a single object set, dnodes have an object number, which is the index of their position in the object set’s array of dnodes. (Because an object number is just the index of the object’s dnode in its object set’s array of dnodes, object numbers are basically always going to be duplicated between object sets (and they’re always relative to an object set). For instance, pretty much every object set is going to have an object number ten, although not all object sets may have enough objects that they have an object number ten thousand. One corollary of this is that if you ask zdb to tell you about a given object number, you have to tell zdb what object set you’re talking about. Usually you do this by telling zdb which ZFS filesystem or dataset you mean.) Each ZFS filesystem has its own object set for objects (and thus dnodes) used in the filesystem. As I discovered yesterday, every ZFS filesystem has a directory hierarchy and it may go many levels deep, but all of this directory hierarchy refers to directories and files using their object number. ZFS organizes and keeps track of filesystems, clones, and snapshots through the DSL (Dataset and Snapshot Layer). The DSL has all sorts of things; DSL directories, DSL datasets, and so on, all of which are objects and many of which refer to object sets (for example, every ZFS filesystem must refer to its current object set somehow). All of these DSL objects are themselves stored as dnodes in another object set, the Meta Object Set, which the uberblock points to. To my surprise, object sets are not stored in the MOS (and as a result do not have ‘object numbers’). Object sets are always referred to directly, without indirection, using a block pointer to the object set’s dnode. (I think object sets are referred to directly so that snapshots can freeze their object set very simply.) The DSL directories and datasets for your pool’s set of filesystems form a tree themselves (each filesystem has a DSL directory and at least one DSL dataset). However, just like in ZFS filesystems, all of the objects in this second tree refer to each other indirectly, by their MOS object number. Just as with files in ZFS filesystems, this level of indirection limits the amount of copy on write updates that ZFS had to do when something changes. PS: If you want to examine MOS objects with zdb, I think you do it with something like ‘zdb -vvv -d ssddata 1’, which will get you object number 1 of the MOS, which is the MOS object directory. If you want to ask zdb about an object in the pool’s root filesystem, use ‘zdb -vvv -d ssddata/ 1’. You can tell which one you’re getting depending on what zdb prints out. If it says ‘Dataset mos [META]’ you’re looking at objects from the MOS; if it says ‘Dataset ssddata [ZPL]’, you’re looking at the pool’s root filesystem (where object number 1 is the ZFS master node). PPS: I was going to write up what changed on a filesystem write, but then I realized that I didn’t know how blocks being allocated and freed are reflected in pool structures. So I’ll just say that I think that ignoring free space management, only four DMU objects get updated; the file itself, the filesystem’s object set, the filesystem’s DSL dataset object, and the MOS. (As usual, doing the research to write this up taught me things that I didn’t know about ZFS.) Digital Ocean ###HardenedBSD Foundation Status On 09 July 2018, the HardenedBSD Foundation Board of Directors held the kick-off meeting to start organizing the Foundation. The following people attended the kick-off meeting: Shawn Webb (in person) George Saylor (in person) Ben Welch (in person) Virginia Suydan (in person) Ben La Monica (phone) Dean Freeman (phone) Christian Severt (phone) We discussed the very first steps that need to be taken to organize the HardenedBSD Foundation as a 501©(3) not-for-profit organization in the US. We determined we could file a 1023EZ instead of the full-blown 1023. This will help speed the process up drastically. The steps are laid out as follows: Register a Post Office Box (PO Box) (completed on 10 Jul 2018). Register The HardenedBSD Foundation as a tax-exempt nonstock corporation in the state of Maryland (started on 10 Jul 2018, submitted on 18 Jul 2018, granted 20 Jul 2018). Obtain a federal tax ID (obtained 20 Jul 2018). Close the current bank account and create a new one using the federal tax ID (completed on 20 Jul 2018). File the 1023EZ paperwork with the federal government (started on 20 Jul 2018). Hire an attorney to help draft the organization bylaws. Each of the steps must be done serially and in order. We added Christian Severt, who is on Emerald Onion’s Board of Directors, to the HardenedBSD Foundation Board of Directors as an advisor. He was foundational in getting Emerald Onion their 501©(3) tax-exempt, not-for-profit status and has really good insight. Additionally, he’s going to help HardenedBSD coordinate hosting services, figuring out the best deals for us. We promoted George Saylor to Vice President and changed Shawn Webb’s title to President and Director. This is to help resolve potential concerns both the state and federal agencies might have with an organization having only a single President role. We hope to be granted our 501©(3) status before the end of the year, though that may be subject to change. We are excited for the formation of the HardenedBSD Foundation, which will open up new opportunities not otherwise available to HardenedBSD. ###More mitigations against speculative execution vulnerabilities Philip Guenther (guenther@) and Bryan Steele (brynet@) have added more mitigations against speculative execution CPU vulnerabilities on the amd64 platform. CVSROOT: /cvs Module name: src Changes by: guenther@cvs.openbsd.org 2018/07/23 11:54:04 Modified files: sys/arch/amd64/amd64: locore.S sys/arch/amd64/include: asm.h cpufunc.h frameasm.h Log message: Do "Return stack refilling", based on the "Return stack underflow" discussion and its associated appendix at https://support.google.com/faqs/answer/7625886 This should address at least some cases of "SpectreRSB" and earlier Spectre variants; more commits to follow. The refilling is done in the enter-kernel-from-userspace and return-to-userspace-from-kernel paths, making sure to do it before unblocking interrupts so that a successive interrupt can't get the CPU to C code without doing this refill. Per the link above, it also does it immediately after mwait, apparently in case the low-power CPU states of idle-via-mwait flush the RSB. ok mlarkin@ deraadt@``` and: ```CVSROOT: /cvs Module name: src Changes by: guenther@cvs.openbsd.org 2018/07/23 20:42:25 Modified files: sys/arch/amd64/amd64: locore.S vector.S vmm_support.S sys/arch/amd64/include: asm.h cpufunc.h Log message: Also do RSB refilling when context switching, after vmexits, and when vmlaunch or vmresume fails. Follow the lead of clang and the intel recommendation and do an lfence after the pause in the speculation-stop path for retpoline, RSB refill, and meltover ASM bits. ok kettenis@ deraadt@``` "Mitigation G-2" for AMD processors: ```CVSROOT: /cvs Module name: src Changes by: brynet@cvs.openbsd.org 2018/07/23 17:25:03 Modified files: sys/arch/amd64/amd64: identcpu.c sys/arch/amd64/include: specialreg.h Log message: Add "Mitigation G-2" per AMD's Whitepaper "Software Techniques for Managing Speculation on AMD Processors" By setting MSR C001_1029[1]=1, LFENCE becomes a dispatch serializing instruction. Tested on AMD FX-4100 "Bulldozer", and Linux guest in SVM vmd(8) ok deraadt@ mlarkin@``` Beastie Bits HardenedBSD will stop supporting 10-STABLE on 10 August 2018 (https://groups.google.com/a/hardenedbsd.org/forum/#!topic/users/xvU0g-g1l5U) GSoC 2018 Reports: Integrate libFuzzer with the Basesystem, Part 2 (https://blog.netbsd.org/tnf/entry/gsoc_2018_reports_integrate_libfuzzer1) ZFS Boot Environments at PBUG (https://vermaden.wordpress.com/2018/07/30/zfs-boot-environments-at-pbug/) Second Editions versus the Publishing Business (https://blather.michaelwlucas.com/archives/3229) Theo de Raadt on "unveil(2) usage in base" (https://undeadly.org/cgi?action=article;sid=20180728063716) rtadvd(8) has been replaced by rad(8) (https://undeadly.org/cgi?action=article;sid=20180724072205) BSD Users Stockholm Meetup #3 (https://www.meetup.com/BSD-Users-Stockholm/events/253447019/) Changes to NetBSD release support policy (https://blog.netbsd.org/tnf/entry/changes_to_netbsd_release_support) The future of HAMMER1 (http://lists.dragonflybsd.org/pipermail/users/2018-July/357832.html) *** Tarsnap Feedback/Questions Rodriguez - A Question (http://dpaste.com/0Y1B75Q#wrap) Shane - About ZFS Mostly (http://dpaste.com/32YGNBY#wrap) Leif - ZFS less than 8gb (http://dpaste.com/2GY6HHC#wrap) Wayne - ZFS vs EMC (http://dpaste.com/17PSCXC#wrap) Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv)
Episode 257: Great NetBSD 8 | BSD Now 257
NetBSD 8.0 available, FreeBSD on Scaleway’s ARM64 VPS, encrypted backups with OpenBSD, Dragonfly server storage upgrade, zpool checkpoints, g2k18 hackathon reports, and more. ##Headlines NetBSD v8.0 Released The NetBSD Project is pleased to announce NetBSD 8.0, the sixteenth major release of the NetBSD operating system. This release brings stability improvements, hundreds of bug fixes, and many new features. Some highlights of the NetBSD 8.0 release are: USB stack rework, USB3 support added. In-kernel audio mixer (audio_system(9)). Reproducible builds (MKREPRO, see mk.conf(5)). Full userland debug information (MKDEBUG, see mk.conf(5)) available. While most install media do not come with them (for size reasons), the debug and xdebug sets can be downloaded and extracted as needed later. They provide full symbol information for all base system and X binaries and libraries and allow better error reporting and (userland) crash analysis. PaX MPROTECT (W^X) memory protection enforced by default on some architectures with fine-grained memory protection and suitable ELF formats: i386, amd64, evbarm, landisk. PaX ASLR (Address Space Layout Randomization) enabled by default on: i386, amd64, evbarm, landisk, sparc64. Position independent executables by default for userland on: i386, amd64, arm, m68k, mips, sh3, sparc64. A new socket layer can(4) has been added for communication of devices on a CAN bus. A special pseudo interface ipsecif(4) for route-based VPNs has been added. Parts of the network stack have been made MP-safe. The kernel option NET_MPSAFE is required to enable this. Hardening of the network stack in general. Various WAPBL (the NetBSD file system “log” option) stability and performance improvements. Specific to i386 and amd64 CPUs: Meltdown mitigation: SVS (Separate Virtual Space), enabled by default. SpectreV2 mitigation: retpoline (support in gcc), used by default for kernels. Other hardware mitigations are also available. SpectreV4 mitigations available for Intel and AMD. PopSS workaround: user access to debug registers is turned off by default. Lazy FPU saving disabled on vulnerable Intel CPUs (“eagerfpu”). SMAP support. Improvement and hardening of the memory layout: W^X, fewer writable pages, better consistency, better performance. (U)EFI bootloader. Many evbarm kernels now use FDT (flat device tree) information (loadable at boot time from an external file) for device configuration, the number of kernels has decreased but the number of boards has vastly increased. Lots of updates to 3rd party software included: GCC 5.5 with support for Address Sanitizer and Undefined Behavior Sanitizer GDB 7.12 GNU binutils 2.27 Clang/LLVM 3.8.1 OpenSSH 7.6 OpenSSL 1.0.2k mdocml 1.14.1 acpica 20170303 ntp 4.2.8p11-o dhcpcd 7.0.6 Lua 5.3.4 ###Running FreeBSD on the ARM64 VPS from Scaleway I’ve been thinking about this 6 since 2017, but only yesterday signed up for an account and played around with the ARM64 offering. Turns out it’s pretty great! KVM boots into UEFI, there’s a local VirtIO disk attached, no NBD junk required. So we can definitely run FreeBSD. I managed to “depenguinate” a running instance, the notes are below. Would be great if Scaleway offered an official image instead :wink: For some reason, unlike on x86 4, mounting additional volumes is not allowed 4 on ARM64 instances. So we’ll have to move the running Linux to a ramdisk using pivotroot and then we can do whatever to our one and only disk. Spin up an instance with Ubuntu Zesty and ssh in. Prepare the system and change the root to a tmpfs: apt install gdisk mount -t tmpfs tmpfs /tmp cp -r /bin /sbin /etc /dev /root /home /lib /run /usr /var /tmp mkdir /tmp/proc /tmp/sys /tmp/oldroot mount /dev/vda /tmp/oldroot mount --make-rprivate / pivotroot /tmp /tmp/oldroot for i in dev proc sys run; do mount --move /oldroot/$i /$i; done systemctl daemon-reload systemctl restart sshd Now reconnect to ssh from a second terminal (note: rm the connection file if you use ControlPersist in ssh config), then exit the old session. Kill the old sshd process, restart or stop the rest of the stuff using the old disk: pkill -f notty sed -ibak 's/RefuseManualStart.$//g' /lib/systemd/system/dbus.service systemctl daemon-reload systemctl restart dbus systemctl daemon-reexec systemctl stop user@0 ntp cron systemd-logind systemctl restart systemd-journald systemd-udevd pkill agetty pkill rsyslogd Check that nothing is touching /oldroot: lsof | grep oldroot There will probably be an old dbus-daemon, kill it. And finally, unmount the old root and overwrite the hard disk with a memstick image: umount -R /oldroot wget https://download.freebsd.org/ftp/snapshots/arm64/aarch64/ISO-IMAGES/12.0/FreeBSD-12.0-CURRENT-arm64-aarch64-20180719-r336479-mini-memstick.img.xz xzcat FreeBSD-12.0-CURRENT-arm64-aarch64-20180719-r336479-mini-memstick.img.xz | dd if=/dev/stdin of=/dev/vda bs=1M (Look for the newest snapshot, don’t copy paste the July 19 link above if you’re reading this in the future. Actually maybe use a release instead of CURRENT…) Now, fix the GPT: move the secondary table to the end of the disk and resize the table. It’s important to resize here, as FreeBSD does not do that and silently creates partitions that won’t persist across reboots gdisk /dev/vda x e s 4 w y And reboot. (You might actually want to hard reboot here: for some reason on the first reboot from Linux, pressing the any-key to enter the prompt in the loader hangs the console for me.) I didn’t have to go into the ESC menu and choose the local disk in the boot manager, it seems to boot from disk automatically. Now we’re in the FreeBSD EFI loader. For some reason, the (recently fixed? 2) serial autodetection from EFI is not working correctly. Or something. So you don’t get console output by default. To fix, you have to run these commands in the boot loader command prompt: set console=comconsole,efi boot Ignore the warning about comconsole not being a valid console. Since there’s at least one (efi) that the loader thinks is valid, it sets the whole variable.) (UPD: shouldn’t be necessary in the next snapshot) Now it’s a regular installation process! When asked about partitioning, choose Shell, and manually add a partition and set up a root filesystem: gpart add -t freebsd-zfs -a 4k -l zroot vtbd0 zpool create -R /mnt -O mountpoint=none -O atime=off zroot /dev/gpt/zroot zfs create -o canmount=off -o mountpoint=none zroot/ROOT zfs create -o mountpoint=/ zroot/ROOT/default zfs create -o mountpoint=/usr zroot/ROOT/default/usr zfs create -o mountpoint=/var zroot/ROOT/default/var zfs create -o mountpoint=/var/log zroot/ROOT/default/var/log zfs create -o mountpoint=/usr/home zroot/home zpool set bootfs=zroot/ROOT/default zroot exit (In this example, I set up ZFS with a beadm-compatible layout which allows me to use Boot Environments.) In the post-install chroot shell, fix some configs like so: echo 'zfsload="YES"' >> /boot/loader.conf echo 'console="comconsole,efi"' >> /boot/loader.conf echo 'vfs.zfs.arcmax="512M"' >> /boot/loader.conf sysrc zfsenable=YES exit (Yeah, for some reason, the loader does not load zfs.ko’s dependency opensolaris.ko automatically here. idk what even. It does on my desktop and laptop.) Now you can reboot into the installed system!! Here’s how you can set up IPv6 (and root’s ssh key) auto configuration on boot: Pkg bootstrap pkg install curl curl https://raw.githubusercontent.com/scaleway/image-tools/master/bases/overlay-common/usr/local/bin/scw-metadata > /usr/local/bin/scw-metadata chmod +x /usr/local/bin/scw-metadata echo '#!/bin/sh' > /etc/rc.local echo 'PATH=/usr/local/bin:$PATH' >> /etc/rc.local echo 'eval $(scw-metadata)' >> /etc/rc.local echo 'echo $SSHPUBLICKEYS0KEY > /root/.ssh/authorizedkeys' >> /etc/rc.local echo 'chmod 0400 /root/.ssh/authorizedkeys' >> /etc/rc.local echo 'ifconfig vtnet0 inet6 $IPV6ADDRESS/$IPV6NETMASK' >> /etc/rc.local echo 'route -6 add default $IPV6GATEWAY' >> /etc/rc.local mkdir /run mkdir /root/.ssh sh /etc/rc.local And to fix incoming TCP connections, configure the DHCP client to change the broadcast address: echo 'interface "vtnet0" { supersede broadcast-address 255.255.255.255; }' >> /etc/dhclient.conf killall dhclient dhclient vtnet0 Other random notes: keep in mind that -CURRENT snapshots come with a debugging kernel by default, which limits syscall performance by a lot, you might want to build your own 2 with config GENERIC-NODEBUG also disable heavy malloc debugging features by running ln -s ‘abort:false,junk:false’ /etc/malloc.conf (yes that’s storing config in a symlink) you can reuse the installer’s partition for swap * Digital Ocean ** http://do.co/bsdnow ###Easy encrypted backups on OpenBSD with base tools Today’s topic is “Encrypted backups” using only OpenBSD base tools. I am planning to write a bigger article later about backups but it’s a wide topic with a lot of software to cover and a lot of explanations about the differents uses cases, needs, issues an solutions. Here I will stick on explaining how to make reliable backups for an OpenBSD system (my laptop). What we need is the dump command (see man 8 dump for its man page). It’s an utility to make a backup for a filesystem, it can only make a backup of one filesystem at a time. On my laptop I only backup /home partition so this solution is suitable for me while still being easy. Dump can do incremental backups, it means that it will only save what changed since the last backup of lower level. If you do not understand this, please refer to the dump man page. What is very interesting with dump is that it honors nodump flag which is an extended attribute of a FFS filesystem. One can use the command chflags nodump /home/solene/Downloads to tells dump not do save that folder (under some circumstances). By default, dump will not save thoses files, EXCEPT for a level 0 backup. Important features of this backup solution: save files with attributes, permissions and flags can recreate a partition from a dump, restore files interactively, from a list or from its inode number (useful when you have files in lost+found) one dump = one file My process is to make a huge dump of level 0 and keep it on a remote server, then, once a week I make a level 1 backup which will contain everything changed since the last dump of level 0, and everyday I do a level 2 backup of my files. The level 2 will contain latest files and the files changing a lot, which are often the most interesting. The level 1 backup is important because it will offload a lot of changes for the level 2. Let me explain: let says my full backup is 60 GB, full of pictures, sources files, GUI applications data files etc… A level 1 backup will contain every new picture, new projects, new GUI files etc… since the full backup, which will produce bigger and bigger dump over time, usually it is only 100 MB to 1GB. As I don’t add new pictures everyday or use new software everyday, the level 2 will take care of most littles changes to my data, like source code edited, little works on files etc… The level 2 backup is really small, I try to keep it under 50 MB so I can easily send it on my remote server everyday. One could you more dump level, up to level 9, but keep in mind that those are incremental. In my case, if I need to restore all my partition, I will need to use level 0, 1 and 2 to get up to latest backup state. If you want to restore a file deleted a few days ago, you need to remember in which level its latest version is. History note: dump was designed to be used with magnetic tapes. See the article for the remainder of the article ##News Roundup Status of DFly server storage upgrades (Matt Dillon) Last month we did some storage upgrades, particularly of internet-facing machines for package and OS distribution. Yesterday we did a number of additional upgrades, described below. All using funds generously donated by everyone! The main repository server received a 2TB SSD to replace the HDDs it was using before. This will improve access to a number of things maintained by this server, including the mail archives, and gives the main repo server more breathing room for repository expansion. Space was at a premium before. Now there’s plenty. Monster, the quad socket opteron which we currently use as the database builder and repository that we export to our public grok service (grok.dragonflybsd.org) received a 512G SSD to add swap space for swapcache, to help cache the grok meta-data. It now has 600GB of swapcache configured. Over the next few weeks we will also be changing the grok updates to ping-pong between the two 4TB data drives it received in the last upgrade so we can do concurrent updates and web accesses without them tripping over each other performance-wise. The main developer box, Leaf, received a 2TB SSD and we are currently in the midst of migrating all the developer accounts in /home and /build from its old HDDs to its new SSD. This machine serves developer repos, developer web stuff, our home page and wiki, etc, so those will become snappier as well. Hard drives are becoming real dinosaurs. We still have a few left from the old days but in terms of active use the only HDDs we feel we really need to keep now are the ones we use for backups and grok data, owing to the amount of storage needed for those functions. Five years ago when we received the blade server that now sits in the colo, we had a small 256G SSD for root on every blade, and everything else used HDDs. To make things operate smoothly, most of that 256G root SSD was assigned to swapcache (200G of it, in fact, in most cases). Even just 2 years ago replacing all those HDDs with SSDs, even just the ones being used to actively serve data and support developers, would have been cost prohibitive. But today it isn’t and the only HDDs we really need anywhere are for backups or certain very large bits of bulk data (aka the grok source repository and index). The way things are going, even the backup drives will probably become SSDs over the next two years. ###iX ad spot OSCON 2018 Recap ###zpool checkpoints In March, to FreeBSD landed a very interesting feature called ‘zpool checkpoints’. Before we jump straight into the topic, let’s take a step back and look at another ZFS feature called ‘snapshot’. Snapshot allows us to create an image of our single file systems. This gives us the option to modify data on the dataset without the fear of losing some data. A very good example of how to use ZFS snapshot is during an upgrade of database schema. Let us consider a situation where we have a few scripts which change our schema. Sometimes we are unable to upgrade in one transaction (for example, when we attempt to alter a table and then update it in single transaction). If our database is on dataset, we can just snapshot it, and if something goes wrong, simply rollback the file system to its previous state. The problem with snapshot is that it works only on a single dataset. If we added some dataset, we wouldn’t then be able to create the snapshot which would rollback that operation. The same with changing the attributes of a dataset. If we change the compression on the dataset, we cannot rollback it. We would need to change that manually. Another interesting problem involves upgrading the whole operating system when we upgrade system with a new ZFS version. What if we start upgrading our dataset and our kernel begins to crash? (If you use FreeBSD, I doubt you will ever have had that experience but still…). If we rollback to the old kernel, there is a chance the dataset will stop working because the new kernel doesn’t know how to use the new features. Zpool checkpoints is the solution to all those problems. Instead of taking a single snapshot of the dataset, we can now take a snapshot of the whole pool. That means we will not only rollback the data but also all the metadata. If we rewind to the checkpoint, all our ZFS properties will be rolled back; the upgrade will be rolledback, and even the creation/deletion of the dataset, and the snapshot, will be rolledback. Zpool Checkpoint has introduced a few simple functions: For a creating checkpoint: zpool checkpoint <pool> Rollbacks state to checkpoint and remove the checkpoint: zpool import -- rewind-to-checkpoint <pool> Mount the pool read only - this does not rollback the data: zpool import --read-only=on --rewind-to-checkpoint Remove the checkpoint zpool checkpoint --discard <pool> or zpool checkpoint -d <pool> With this powerful feature we need to remember some safety rules: Scrub will work only on data that isn’t in checkpool. You can’t remove vdev if you have a checkpoint. You can’t split mirror. Reguid will not work either. Create a checkpoint when one of the disks is removed… For me, this feature is incredibly useful, especially when upgrading an operating system, or when I need to experiment with additional data sets. If you speak Polish, I have some additional information for you. During the first Polish BSD user group meeting, I had the opportunity to give a short talk about this feature. Here you find the video of that talk, and here is the slideshow. I would like to offer my thanks to Serapheim Dimitropoulos for developing this feature, and for being so kind in sharing with me so many of its intricacies. If you are interested in knowing more about the technical details of this feature, you should check out Serapheim’s blog, and his video about checkpoints. ###g2k18 Reports g2k18 hackathon report: Ingo Schwarze on sed(1) bugfixing with Martijn van Duren, and about other small userland stuff g2k18 hackathon report: Kenneth Westerback on dhcpd(8) fixes, disklabel(8) refactoring and more g2k18 Hackathon Report: Marc Espie on ports and packages progress g2k18 hackathon report: Antoine Jacoutot on porting g2k18 hackathon report: Matthieu Herrb on font caches and xenodm g2k18 hackathon report: Florian Obser on rtadvd(8) -> rad(8) progress (actually, rewrite) g2k18 Hackathon Report: Klemens Nanni on improvements to route(8), pfctl(8), and mount(2) g2k18 hackathon report: Carlos Cardenas on vmm/vmd progress, LACP g2k18 hackathon report: Claudio Jeker on OpenBGPD developments Picture of the last day of the g2k18 hackathon in Ljubljana, Slovenia ##Beastie Bits Something blogged (on pkgsrcCon 2018) GSoC 2018 Reports: Configuration files versioning in pkgsrc, Part 1 There should be a global ‘awareness’ week for developers Polish BSD User Group – Upcoming Meeting: Aug 9th 2018 London BSD User Group – Upcoming Meeting: Aug 14th 2018 Phillip Smith’s collection of reasons why ZFS is better so that he does not have to repeat himself all the time EuroBSDCon 2018: Sept 20-23rd in Romania – Register NOW! MeetBSD 2018: Oct 19-20 in Santa Clara, California. Call for Papers closes on Aug 12 Tarsnap ##Feedback/Questions Dale - L2ARC recommendations & drive age question Todd - ZFS & S3 efraim - License Poem Henrick - Yet another ZFS question Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Episode 256: Because Computers | BSD Now 2^8
FreeBSD ULE vs. Linux CFS, OpenBSD on Tuxedo InfinityBook, how zfs diff reports filenames efficiently, why choose FreeBSD over Linux, PS4 double free exploit, OpenBSD’s wifi autojoin, and FreeBSD jails the hard way. Win Celebrate our 256th episode with us. You can win a Mogics Power Bagel (not sponsored). To enter, go find the 4 episodes we did in December of 2017. In the opening, find the 4 letters in the bookshelf behind me. They spell different words in each of the 4 episodes. Send us these words in order to feedback@bsdnow.tv with the subject “bsdnow256” until August 8th, 2018 18:00 UTC and we’ll randomly draw the winner on the live show. We’ll then contact you to ship the item. Only one item to win. All decisions are final. Better luck next time. Headlines Battle of the Schedulers: FreeBSD ULE vs. Linux CFS Introduction This paper analyzes the impact on application performance of the design and implementation choices made in two widely used open-source schedulers: ULE, the default FreeBSD scheduler, and CFS, the default Linux scheduler. We compare ULE and CFS in otherwise identical circumstances. We have ported ULE to Linux, and use it to schedule all threads that are normally scheduled by CFS. We compare the performance of a large suite of applications on the modified kernel running ULE and on the standard Linux kernel running CFS. The observed performance differences are solely the result of scheduling decisions, and do not reflect differences in other subsystems between FreeBSD and Linux. There is no overall winner. On many workloads the two schedulers perform similarly, but for some workloads there are significant and even surprising differences. ULE may cause starvation, even when executing a single application with identical threads, but this starvation may actually lead to better application performance for some workloads. The more complex load balancing mechanism of CFS reacts more quickly to workload changes, but ULE achieves better load balance in the long run. Operating system kernel schedulers are responsible for maintaining high utilization of hardware resources (CPU cores, memory, I/O devices) while providing fast response time to latency-sensitive applications. They have to react to workload changes, and handle large numbers of cores and threads with minimal overhead [12]. This paper provides a comparison between the default schedulers of two of the most widely deployed open-source operating systems: the Completely Fair Scheduler (CFS) used in Linux, and the ULE scheduler used in FreeBSD. Our goal is not to declare an overall winner. In fact, we find that for some workloads ULE is better and for others CFS is better. Instead, our goal is to illustrate how differences in the design and the implementation of the two schedulers are reflected in application performance under different workloads. ULE and CFS are both designed to schedule large numbers of threads on large multicore machines. Scalability considerations have led both schedulers to adopt per-core run-queues. On a context switch, a core accesses only its local run-queue to find the next thread to run. Periodically and at select times, e.g., when a thread wakes up, both ULE and CFS perform load balancing, i.e., they try to balance the amount of work waiting in the run-queues of different cores. ULE and CFS, however, differ greatly in their design and implementation choices. FreeBSD ULE is a simple scheduler (2,950 lines of code in FreeBSD 11.1), while Linux CFS is much more complex (17,900 lines of code in the latest LTS Linux kernel, Linux 4.9). FreeBSD run-queues are FIFO. For load balancing, FreeBSD strives to even out the number of threads per core. In Linux, a core decides which thread to run next based on prior execution time, priority, and perceived cache behavior of the threads in its runqueue. Instead of evening out the number of threads between cores, Linux strives to even out the average amount of pending work. Performance analysis We now analyze the impact of the per-core scheduling on the performance of 37 applications. We define “performance” as follows: for database workloads and NAS applications, we compare the number of operations per second, and for the other applications we compare “execution time”. The higher the “performance”, the better a scheduler performs. Figure 5 presents the performance difference between CFS and ULE on a single core, with percentages above 0 meaning that the application executes faster with ULE than CFS. Overall, the scheduler has little influence on most workloads. Indeed, most applications use threads that all perform the same work, thus both CFS and ULE endup scheduling all of the threads in a round-robin fashion. The average performance difference is 1.5%, in favor of ULE. Still, scimark is 36% slower on ULE than CFS, and apache is 40% faster on ULE than CFS. Scimark is a single-threaded Java application. It launches one compute thread, and the Java runtime executes other Java system threads in the background (for the garbage collector, I/O, etc.). When the application is executed with ULE, the compute thread can be delayed, because Java system threads are considered interactive and get priority over the computation thread. The apache workload consists of two applications: the main server (httpd) running 100 threads, and ab, a single-threaded load injector. The performance difference between ULE and CFS is explained by different choices regarding thread preemption. In ULE, full preemption is disabled, while CFS preempts the running thread when the thread that has just been woken up has a vruntime that is much smaller than the vruntime of the currently executing thread (1ms difference in practice). In CFS, ab is preempted 2 million times during the benchmark, while it never preempted with ULE. This behavior is explained as follows: ab starts by sending 100 requests to the httpd server, and then waits for the server to answer. When ab is woken up, it checks which requests have been processed and sends new requests to the server. Since ab is single-threaded, all requests sent to the server are sent sequentially. In ULE, ab is able to send as many new requests as it has received responses. In CFS, every request sent by ab wakes up a httpd thread, which preempts ab. Conclusion Scheduling threads on a multicore machine is hard. In this paper, we perform a fair comparison of the design choices of two widely used schedulers: the ULE scheduler from FreeBSD and CFS from Linux. We show that they behave differently even on simple workloads, and that no scheduler performs better than the other on all workloads. OpenBSD 6.3 on Tuxedo InfinityBook Disclaimer: I came across the Tuxedo Computers InfinityBook last year at the Open! Conference where Tuxedo had a small booth. Previously they came to my attention since they’re a member of the OSB Alliance on whose board I’m a member. Furthermore Tuxedo Computers are a sponsor of the OSBAR which I’m part of the organizational team. OpenBSD on the Tuxedo InfinityBook I’ve asked the guys over at Tuxedo Computers whether they would be interested to have some tests with *BSD done and that I could test drive one of their machines and give feedback on what works and what does not - and possibly look into it.+ Within a few weeks they shipped me a machine and last week the InfinityBook Pro 14” arrived. Awesome. Thanks already to the folks at Tuxedo Computers. The machine arrived accompanied by lot’s of swag :) The InfinityBook is a very nice machine and allows a wide range of configuration. The configuration that was shipped to me: Intel Core i7-8550U 1x 16GB RAM 2400Mhz Crucial Ballistix Sport LT 250 GB Samsung 860 EVO (M.2 SATAIII) I used a USB-stick to boot install63.fs and re-installed the machine with OpenBSD. Full dmesg. The installation went flawlessly, the needed intel firmware is being installed after installation automatically via fw_update(1). Out of the box the graphics works and once installed the machine presents the login. Video When X starts the display is turned off for some reason. You will need to hit fn+f12 (the key with the moon on it) then the display will go on. Aside from that little nit, X works just fine and presents one the expected resolution. External video is working just fine as well. Either via hdmi output or via the mini displayport connector. The buttons for adjusting brightness (fn+f8 and fn+f9) are not working. Instead one has to use wsconsctl(8) to adjust the brightness. Networking The infinityBook has built-in ethernet, driven by re(4) And for the wireless interface the iwm(4) driver is being used. Both work as expected. ACPI Neither suspend nor hibernate work. Reporting of battery status is bogus as well. Some of the keyboard function keys work: LCD on/off works (fn+f2) Keyboard backlight dimming works (fn+f4) Volume (fn+f5 / fn+f6) works Sound The azalia chipset is being used for audio processing. Works as expected, volume can be controlled via buttons (fn+f5, fn+f6) or via mixerctl. Touchpad Can be controlled via wsconsctl(8). So far I must say, that the InfinityBook makes a nice machine - and I’m enjoying working with it. iXsystems iXsystems - Its all NAS How ZFS makes things like ‘zfs diff’ report filenames efficiently As a copy on write (file)system, ZFS can use the transaction group (txg) numbers that are embedded in ZFS block pointers to efficiently find the differences between two txgs; this is used in, for example, ZFS bookmarks. However, as I noted at the end of my entry on block pointers, this doesn’t give us a filesystem level difference; instead, it essentially gives us a list of inodes (okay, dnodes) that changed. In theory, turning an inode or dnode number into the path to a file is an expensive operation; you basically have to search the entire filesystem until you find it. In practice, if you’ve ever run ‘zfs diff’, you’ve likely noticed that it runs pretty fast. Nor is this the only place that ZFS quickly turns dnode numbers into full paths, as it comes up in ‘zpool status’ reports about permanent errors. At one level, zfs diff and zpool status do this so rapidly because they ask the ZFS code in the kernel to do it for them. At another level, the question is how the kernel’s ZFS code can be so fast. The interesting and surprising answer is that ZFS cheats, in a way that makes things very fast when it works and almost always works in normal filesystems and with normal usage patterns. The cheat is that ZFS dnodes record their parent’s object number. If you’re familiar with the twists and turns of Unix filesystems, you’re now wondering how ZFS deals with hardlinks, which can cause a file to be in several directories at once and so have several parents (and then it can be removed from some of the directories). The answer is that ZFS doesn’t; a dnode only ever tracks a single parent, and ZFS accepts that this parent information can be inaccurate. I’ll quote the comment in zfsobjto_pobj: When a link is removed [the file’s] parent pointer is not changed and will be invalid. There are two cases where a link is removed but the file stays around, when it goes to the delete queue and when there are additional links. Before I get into the details, I want to say that I appreciate the brute force elegance of this cheat. The practical reality is that most Unix files today don’t have extra hardlinks, and when they do most hardlinks are done in ways that won’t break ZFS’s parent stuff. The result is that ZFS has picked an efficient implementation that works almost all of the time; in my opinion, the great benefit we get from having it around are more than worth the infrequent cases where it fails or malfunctions. Both zfs diff and having filenames show up in zpool status permanent error reports are very useful (and there may be other cases where this gets used). The current details are that any time you hardlink a file to somewhere or rename it, ZFS updates the file’s parent to point to the new directory. Often this will wind up with a correct parent even after all of the dust settles; for example, a common pattern is to write a file to an initial location, hardlink it to its final destination, and then remove the initial location version. In this case, the parent will be correct and you’ll get the right name. News Roundup What is FreeBSD? Why Should You Choose It Over Linux? Not too long ago I wondered if and in what situations FreeBSD could be faster than Linux and we received a good amount of informative feedback. So far, Linux rules the desktop space and FreeBSD rules the server space. In the meantime, though, what exactly is FreeBSD? And at what times should you choose it over a GNU/Linux installation? Let’s tackle these questions. FreeBSD is a free and open source derivative of BSD (Berkeley Software Distribution) with a focus on speed, stability, security, and consistency, among other features. It has been developed and maintained by a large community ever since its initial release many years ago on November 1, 1993. BSD is the version of UNIX® that was developed at the University of California in Berkeley. And being a free and open source version, “Free” being a prefix to BSD is a no-brainer. What’s FreeBSD Good For? FreeBSD offers a plethora of advanced features and even boasts some not available in some commercial Operating Systems. It makes an excellent Internet and Intranet server thanks to its robust network services that allow it to maximize memory and work with heavy loads to deliver and maintain good response times for thousands of simultaneous user processes. FreeBSD runs a huge number of applications with ease. At the moment, it has over 32,000 ported applications and libraries with support for desktop, server, and embedded environments. with that being said, let me also add that FreeBSD is excellent for working with advanced embedded platforms. Mail and web appliances, timer servers, routers, MIPS hardware platforms, etc. You name it! FreeBSD is available to install in several ways and there are directions to follow for any method you want to use; be it via CD-ROM, over a network using NFS or FTP, or DVD. FreeBSD is easy to contribute to and all you have to do is to locate the section of the FreeBSD code base to modify and carefully do a neat job. Potential contributors are also free to improve on its artwork and documentation, among other project aspects. FreeBSD is backed by the FreeBSD Foundation, a non-profit organization that you can contribute to financially and all direct contributions are tax deductible. FreeBSD’s license allows users to incorporate the use of proprietary software which is ideal for companies interested in generating revenues. Netflix, for example, could cite this as one of the reasons for using FreeBSD servers. Why Should You Choose It over Linux? From what I’ve gathered about both FreeBSD and Linux, FreeBSD has a better performance on servers than Linux does. Yes, its packaged applications are configured to offer better a performance than Linux and it is usually running fewer services by default, there really isn’t a way to certify which is faster because the answer is dependent on the running hardware and applications and how the system is tuned. FreeBSD is reportedly more secure than Linux because of the way the whole project is developed and maintained. Unlike with Linux, the FreeBSD project is controlled by a large community of developers around the world who fall into any of these categories; core team, contributors, and committers. FreeBSD is much easier to learn and use because there aren’t a thousand and one distros to choose from with different package managers, DEs, etc. FreeBSD is more convenient to contribute to because it is the entire OS that is preserved and not just the kernel and a repo as is the case with Linux. You can easily access all of its versions since they are sorted by release numbers. Apart from the many documentations and guides that you can find online, FreeBSD has a single official documentation wherein you can find the solution to virtually any issue you will come across. So, you’re sure to find it resourceful. FreeBSD has close to no software issues compared to Linux because it has Java, is capable of running Windows programs using Wine, and can run .NET programs using Mono. FreeBSD’s ports/packages system allows you to compile software with specific configurations, thereby avoiding conflicting dependency and version issues. Both the FreeBSD and GNU/Linux project are always receiving updates. The platform you decide to go with is largely dependent on what you want to use it for, your technical know-how, willingness to learn new stuff, and ultimately your preference. What is your take on the topic? For what reasons would you choose FreeBSD over Linux if you would? Let us know what you think about both platforms in the comments section below. PS4 5.05 BPF Double Free Kernel Exploit Writeup Introduction Welcome to the 5.0x kernel exploit write-up. A few months ago, a kernel vulnerability was discovered by qwertyoruiopz and an exploit was released for BPF which involved crafting an out-of-bounds (OOB) write via use-after-free (UAF) due to the lack of proper locking. It was a fun bug, and a very trivial exploit. Sony then removed the write functionality from BPF, so that exploit was patched. However, the core issue still remained (being the lack of locking). A very similar race condition still exists in BPF past 4.55, which we will go into detail below on. The full source of the exploit can be found here. This bug is no longer accessible however past 5.05 firmware, because the BPF driver has finally been blocked from unprivileged processes - WebKit can no longer open it. Sony also introduced a new security mitigation in 5.0x firmwares to prevent the stack pointer from pointing into user space, however we’ll go more in detail on this a bit further down. Assumptions Some assumptions are made of the reader’s knowledge for the writeup. The avid reader should have a basic understanding of how memory allocators work - more specifically, how malloc() and free() allocate and deallocate memory respectively. They should also be aware that devices can be issued commands concurrently, as in, one command could be received while another one is being processed via threading. An understanding of C, x86, and exploitation basics is also very helpful, though not necessarily required. Background This section contains some helpful information to those newer to exploitation, or are unfamiliar with device drivers, or various exploit techniques such as heap spraying and race conditions. Feel free to skip to the “A Tale of Two Free()'s” section if you’re already familiar with this material. What Are Drivers? There are a few ways that applications can directly communicate with the operating system. One of which is system calls, which there are over 600 of in the PS4 kernel, ~500 of which are FreeBSD - the rest are Sony-implemented. Another method is through something called “Device Drivers”. Drivers are typically used to bridge the gap between software and hardware devices (usb drives, keyboard/mouse, webcams, etc) - though they can also be used just for software purposes. There are a few operations that a userland application can perform on a driver (if it has sufficient permissions) to interface with it after opening it. In some instances, one can read from it, write to it, or in some cases, issue more complex commands to it via the ioctl() system call. The handlers for these commands are implemented in kernel space - this is important, because any bugs that could be exploited in an ioctl handler can be used as a privilege escalation straight to ring0 - typically the most privileged state. Drivers are often the more weaker points of an operating system for attackers, because sometimes these drivers are written by developers who don’t understand how the kernel works, or the drivers are older and thus not wise to newer attack methods. The BPF Device Driver If we take a look around inside of WebKit’s sandbox, we’ll find a /dev directory. While this may seem like the root device driver path, it’s a lie. Many of the drivers that the PS4 has are not exposed to this directory, but rather only ones that are needed for WebKit’s operation (for the most part). For some reason though, BPF (aka. the “Berkely Packet Filter”) device is not only exposed to WebKit’s sandbox - it also has the privileges to open the device as R/W. This is very odd, because on most systems this driver is root-only (and for good reason). If you want to read more into this, refer to my previous write-up with 4.55FW. What Are Packet Filters? Below is an excerpt from the 4.55 bpfwrite writeup. Since the bug is directly in the filter system, it is important to know the basics of what packet filters are. Filters are essentially sets of pseudo-instructions that are parsed by bpf_filter() (which are ran when packets are received). While the pseudo-instruction set is fairly minimal, it allows you to do things like perform basic arithmetic operations and copy values around inside it’s buffer. Breaking down the BPF VM in it’s entirety is far beyond the scope of this write-up, just know that the code produced by it is ran in kernel mode - this is why read/write access to /dev/bpf should be privileged. Race Conditions Race conditions occur when two processes/threads try to access a shared resource at the same time without mutual exclusion. The problem was ultimately solved by introducing concepts such as the “mutex” or “lock”. The idea is when one thread/process tries to access a resource, it will first acquire a lock, access it, then unlock it once it’s finished. If another thread/process tries to access it while the other has the lock, it will wait until the other thread is finished. This works fairly well - when it’s used properly. Locking is hard to get right, especially when you try to implement fine-grained locking for performance. One single instruction or line of code outside the locking window could introduce a race condition. Not all race conditions are exploitable, but some are (such as this one) - and they can give an attacker very powerful bugs to work with. Heap Spraying The process of heap spraying is fairly simple - allocate a bunch of memory and fill it with controlled data in a loop and pray your allocation doesn’t get stolen from underneath you. It’s a very useful technique when exploiting something such as a use-after-free(), as you can use it to get controlled data into your target object’s backing memory. By extension, it’s useful to do this for a double free() as well, because once we have a stale reference, we can use a heap spray to control the data. Since the object will be marked “free” - the allocator will eventually provide us with control over this memory, even though something else is still using it. That is, unless, something else has already stolen the pointer from you and corrupts it - then you’ll likely get a system crash, and that’s no fun. This is one factor that adds to the variance of exploits, and typically, the smaller the object, the more likely this is to happen. Follow the link to read more of the article DigitalOcean http://do.co/bsdnow OpenBSD gains Wi-Fi “auto-join” In a change which is bound to be welcomed widely, -current has gained “auto-join” for Wi-Fi networks. Peter Hessler (phessler@) has been working on this for quite some time and he wrote about it in his p2k18 hackathon report. He has committed the work from the g2k18 hackathon in Ljubljana: CVSROOT: /cvs Module name: src Changes by: phessler@cvs.openbsd.org 2018/07/11 14:18:09 Modified files: sbin/ifconfig : ifconfig.8 ifconfig.c sys/net80211 : ieee80211ioctl.c ieee80211ioctl.h ieee80211node.c ieee80211node.h ieee80211_var.h Log message: Introduce 'auto-join' to the wifi 802.11 stack. This allows a system to remember which ESSIDs it wants to connect to, any relevant security configuration, and switch to it when the network we are currently connected to is no longer available. Works when connecting and switching between WPA2/WPA1/WEP/clear encryptions. example hostname.if: join home wpakey password join work wpakey mekmitasdigoat join open-lounge join cafe wpakey cafe2018 join "wepnetwork" nwkey "12345" dhcp inet6 autoconf up OK stsp@ reyk@ and enthusiasm from every hackroom I've been in for the last 3 years The usage should be clear from the commit message, but basically you ‘join’ all the networks you want to auto-join as you would previously use ‘nwid’ to connect to one specific network. Then the kernel will join the network that’s actually in range and do the rest automagically for you. When you move out of range of that network you lose connectivity until you come in range of the original (where things will continue to work as you’ve been used to) or one of the other networks (where you will associate and then get a new lease). Thanks to Peter for working on this feature - something many a Wi-Fi using OpenBSD user will be able to benefit from. FreeBSD Jails the hard way There are many great options for managing FreeBSD Jails. iocage, warden and ez-jail aim to streamline the process and make it quick an easy to get going. But sometimes the tools built right into the OS are overlooked. This post goes over what is involved in creating and managing jails using only the tools built into FreeBSD. For this guide, I’m going to be putting my jails in /usr/local/jails. I’ll start with a very simple, isolated jail. Then I’ll go over how to use ZFS snapshots, and lastly nullfs mounts to share the FreeBSD base files with multiple jails. I’ll also show some examples of how to use the templating power of jail.conf to apply similar settings to all your jails. Full Jail Make a directory for the jail, or a zfs dataset if you prefer. Download the FreeBSD base files, and any other parts of FreeBSD you want. In this example I’ll include the 32 bit libraries as well. Update your FreeBSD base install. Verify your download. We’re downloading these archives over FTP after all, we should confirm that this download is valid and not tampered with. The freebsd-update IDS command verifies the installation using a PGP key which is in your base system, which was presumably installed with an ISO that you verified using the FreeBSD signed checksums. Admittedly this step is a bit of paranoia, but I think it’s prudent. Make sure you jail has the right timezone and dns servers and a hostname in rc.conf. Edit jail.conf with the details about your jail. Start and login to your jail. 11 commands and a config file, but this is the most tedious way to make a jail. With a little bit of templating it can be even easier. So I’ll start by making a template. Making a template is basically the same as steps 1, 2 and 3 above, but with a different destination folder, I’ll condense them here. Creating a template Create a template or a ZFS dataset. If you’d like to use the zfs clone method of deploying templates, you’ll need to create a zfs dataset instead of a folder. Update your template with freebsd-update. Verify your install And that’s it, now you have a fully up to date jail template. If you’ve made this template with zfs, you can easily deploy it using zfs snapshots. Deploying a template with ZFS snapshots Create a snapshot. My last freebsd-update to my template brought it to patch level 17, so I’ll call my snapshot p10. Clone the snapshot to a new jail. Configure the jail hostname. Add the jail definition to jail.conf, make sure you have the global jail settings from jail.conf listed in the fulljail example. Start the jail. The downside with the zfs approach is that each jail is now a fully independent, and if you need to update your jails, you have to update them all individually. By sharing a template using nullfs mounts you can have only one copy of the base system that only needs to be updated once. Follow the link to see the rest of the article about Thin jails using NullFS mounts Simplifying jail.conf Hopefully this has helped you understand the process of how to create and manage FreeBSD jails without tools that abstract away all the details. Those tools are often quite useful, but there is always benefit in learning to do things the hard way. And in this case, the hard way doesn’t seem to be that hard after all. Beastie Bits Meetup in Zurich #4, July edition (July 19) – Which you likely missed, but now you know to look for the August edition! The next two BSD-PL User group meetings in Warsaw have been scheduled for July 30th and Aug 9th @ 1830 CEST – Submit your topic proposals now Linux Geek Books - Humble Bundle Extend loader(8) geli support to all architectures and all disk-like devices Upgrading from a bootpool to a single encrypted pool – skip the gptzfsboot part, and manually update your EFI partition with loader.efi The pkgsrc 2018Q2 for Illumos is available with 18500+ binary packages NetBSD ARM64 Images Available with SMP for RPi3 / NanoPi / Pine64 Boards Recently released CDE 2.3.0 running on Tribblix (Illumos) An Interview With Tech & Science Fiction Author Michael W Lucas A reminder : MeetBSD CFP EuroBSDCon talk acceptances have gone out, and once the tutorials are confirmed, registration will open. That will likely have happened by time you see this episode, so go register! See you in Romania Tarsnap Feedback/Questions Wilyarti - Adblocked on FreeBSD Continued… Andrew - A Question and a Story Matthew - Thanks Brian - PCI-E Controller Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Episode 255: What Are You Pointing At | BSD Now 255
What ZFS blockpointers are, zero-day rewards offered, KDE on FreeBSD status, new FreeBSD core team, NetBSD WiFi refresh, poor man’s CI, and the power of Ctrl+T. ##Headlines What ZFS block pointers are and what’s in them I’ve mentioned ZFS block pointers in the past; for example, when I wrote about some details of ZFS DVAs, I said that DVAs are embedded in block pointers. But I’ve never really looked carefully at what is in block pointers and what that means and implies for ZFS. The very simple way to describe a ZFS block pointer is that it’s what ZFS uses in places where other filesystems would simply put a block number. Just like block numbers but unlike things like ZFS dnodes, a block pointer isn’t a separate on-disk entity; instead it’s an on disk data format and an in memory structure that shows up in other things. To quote from the (draft and old) ZFS on-disk specification (PDF): A block pointer (blkptr_t) is a 128 byte ZFS structure used to physically locate, verify, and describe blocks of data on disk. Block pointers are embedded in any ZFS on disk structure that points directly to other disk blocks, both for data and metadata. For instance, the dnode for a file contains block pointers that refer to either its data blocks (if it’s small enough) or indirect blocks, as I saw in this entry. However, as I discovered when I paid attention, most things in ZFS only point to dnodes indirectly, by giving their object number (either in a ZFS filesystem or in pool-wide metadata). So what’s in a block pointer itself? You can find the technical details for modern ZFS in spa.h, so I’m going to give a sort of summary. A regular block pointer contains: various metadata and flags about what the block pointer is for and what parts of it mean, including what type of object it points to. Up to three DVAs that say where to actually find the data on disk. There can be more than one DVA because you may have set the copies property to 2 or 3, or this may be metadata (which normally has two copies and may have more for sufficiently important metadata). The logical size (size before compression) and ‘physical’ size (the nominal size after compression) of the disk block. The physical size can do odd things and is not necessarily the asize (allocated size) for the DVA(s). The txgs that the block was born in, both logically and physically (the physical txg is apparently for dva[0]). The physical txg was added with ZFS deduplication but apparently also shows up in vdev removal. The checksum of the data the block pointer describes. This checksum implicitly covers the entire logical size of the data, and as a result you must read all of the data in order to verify it. This can be an issue on raidz vdevs or if the block had to use gang blocks. Just like basically everything else in ZFS, block pointers don’t have an explicit checksum of their contents. Instead they’re implicitly covered by the checksum of whatever they’re embedded in; the block pointers in a dnode are covered by the overall checksum of the dnode, for example. Block pointers must include a checksum for the data they point to because such data is ‘out of line’ for the containing object. (The block pointers in a dnode don’t necessarily point straight to data. If there’s more than a bit of data in whatever the dnode covers, the dnode’s block pointers will instead point to some level of indirect block, which itself has some number of block pointers.) There is a special type of block pointer called an embedded block pointer. Embedded block pointers directly contain up to 112 bytes of data; apart from the data, they contain only the metadata fields and a logical birth txg. As with conventional block pointers, this data is implicitly covered by the checksum of the containing object. Since block pointers directly contain the address of things on disk (in the form of DVAs), they have to change any time that address changes, which means any time ZFS does its copy on write thing. This forces a change in whatever contains the block pointer, which in turn ripples up to another block pointer (whatever points to said containing thing), and so on until we eventually reach the Meta Object Set and the uberblock. How this works is a bit complicated, but ZFS is designed to generally make this a relatively shallow change with not many levels of things involved (as I discovered recently). As far as I understand things, the logical birth txg of a block pointer is the transaction group in which the block pointer was allocated. Because of ZFS’s copy on write principle, this means that nothing underneath the block pointer has been updated or changed since that txg; if something changed, it would have been written to a new place on disk, which would have forced a change in at least one DVA and thus a ripple of updates that would update the logical birth txg. However, this doesn’t quite mean what I used to think it meant because of ZFS’s level of indirection. If you change a file by writing data to it, you will change some of the file’s block pointers, updating their logical birth txg, and you will change the file’s dnode. However, you won’t change any block pointers and thus any logical birth txgs for the filesystem directory the file is in (or anything else up the directory tree), because the directory refers to the file through its object number, not by directly pointing to its dnode. You can still use logical birth txgs to efficiently find changes from one txg to another, but you won’t necessarily get a filesystem level view of these changes; instead, as far as I can see, you will basically get a view of what object(s) in a filesystem changed (effectively, what inode numbers changed). (ZFS has an interesting hack to make things like ‘zfs diff’ work far more efficiently than you would expect in light of this, but that’s going to take yet another entry to cover.) ###Rewards of Up to $500,000 Offered for FreeBSD, OpenBSD, NetBSD, Linux Zero-Days Exploit broker Zerodium is offering rewards of up to $500,000 for zero-days in UNIX-based operating systems like OpenBSD, FreeBSD, NetBSD, but also for Linux distros such as Ubuntu, CentOS, Debian, and Tails. The offer, first advertised via Twitter earlier this week, is available as part of the company’s latest zero-day acquisition drive. Zerodium is known for buying zero-days and selling them to government agencies and law enforcement. The company runs a regular zero-day acquisition program through its website, but it often holds special drives with more substantial rewards when it needs zero-days of a specific category. BSD zero-day rewards will be on par with Linux payouts The US-based company held a previous drive with increased rewards for Linux zero-days in February, with rewards going as high as $45,000. In another zero-day acquisition drive announced on Twitter this week, the company said it was looking again for Linux zero-days, but also for exploits targeting BSD systems. This time around, rewards can go up to $500,000, for the right exploit. Zerodium told Bleeping Computer they’ll be aligning the temporary rewards for BSD systems with their usual payouts for Linux distros. The company’s usual payouts for Linux privilege escalation exploits can range from $10,000 to $30,000. Local privilege escalation (LPE) rewards can even reach $100,000 for “an exploit with an exceptional quality and coverage,” such as, for example, a Linux kernel exploit affecting all major distributions. Payouts for Linux remote code execution (RCE) exploits can bring in from $50,000 to $500,000 depending on the targeted software/service and its market share. The highest rewards are usually awarded for LPEs and RCEs affecting CentOS and Ubuntu distros. Zero-day price varies based on exploitation chain The acquisition price of a submitted zero-day is directly tied to its requirements in terms of user interaction (no click, one click, two clicks, etc.), Zerodium said. Other factors include the exploit reliability, its success rate, the number of vulnerabilities chained together for the final exploit to work (more chained bugs means more chances for the exploit to break unexpectedly), and the OS configuration needed for the exploit to work (exploits are valued more if they work against default OS configs). Zero-days in servers “can reach exceptional amounts” “Price difference between systems is mostly driven by market shares,” Zerodium founder Chaouki Bekrar told Bleeping Computer via email. Asked about the logic behind these acquisition drives that pay increased rewards, Bekrar told Bleeping Computer the following: "Our aim is to always have, at any time, two or more fully functional exploits for every major software, hardware, or operating systems, meaning that from time to time we would promote a specific software/system on our social media to acquire new codes and strengthen our existing capabilities or extend them.” “We may also react to customers’ requests and their operational needs,” Bekrar said. It’s becoming a crowded market Since Zerodium drew everyone’s attention to the exploit brokerage market in 2015, the market has gotten more and more crowded, but also more sleazy, with some companies being accused of selling zero-days to government agencies in countries with oppressive or dictatorial regimes, where they are often used against political oponents, journalists, and dissidents, instead of going after real criminals. The latest company who broke into the zero-day brokerage market is Crowdfense, who recently launched an acquisition program with prizes of $10 million, of which it already paid $4.5 million to researchers. Twitter Announcement Digital Ocean http://do.co/bsdnow ###KDE on FreeBSD – June 2018 The KDE-FreeBSD team (a half-dozen hardy individuals, with varying backgrounds and varying degrees of involvement depending on how employment is doing) has a status message in the #kde-freebsd channel on freenode. Right now it looks like this: http://FreeBSD.kde.org | Bleeding edge http://FreeBSD.kde.org/area51.php | Released: Qt 5.10.1, KDE SC 4.14.3, KF5 5.46.0, Applications 18.04.1, Plasma-5.12.5, Kdevelop-5.2.1, Digikam-5.9.0 It’s been a while since I wrote about KDE on FreeBSD, what with Calamares and third-party software happening as well. We’re better at keeping the IRC topic up-to-date than a lot of other sources of information (e.g. the FreeBSD quarterly reports, or the f.k.o website, which I’ll just dash off and update after writing this). In no particular order: Qt 5.10 is here, in a FrankenEngine incarnation: we still use WebEnging from Qt 5.9 because — like I’ve said before — WebEngine is such a gigantic pain in the butt to update with all the necessary patches to get it to compile. Our collection of downstream patches to Qt 5.10 is growing, slowly. None of them are upstreamable (e.g. libressl support) though. KDE Frameworks releases are generally pushed to ports within a week or two of release. Actually, now that there is a bigger stack of KDE software in FreeBSD ports the updates take longer because we have to do exp-runs. Similarly, Applications and Plasma releases are reasonably up-to-date. We dodged a bullet by not jumping on Plasma 5.13 right away, I see. Tobias is the person doing almost all of the drudge-work of these updates, he deserves a pint of something in Vienna this summer. The freebsd.kde.org website has been slightly updated; it was terribly out-of-date. So we’re mostly-up-to-date, and mostly all packaged up and ready to go. Much of my day is spent in VMs packaged by other people, but it’s good to have a full KDE developer environment outside of them as well. (PS. Gotta hand it to Tomasz for the amazing application for downloading and displaying a flamingo … niche usecases FTW) ##News Roundup New FreeBSD Core Team Elected Active committers to the project have elected your tenth FreeBSD Core Team. Allan Jude (allanjude) Benedict Reuschling (bcr) Brooks Davis (brooks) Hiroki Sato (hrs) Jeff Roberson (jeff) John Baldwin (jhb) Kris Moore (kmoore) Sean Chittenden (seanc) Warner Losh (imp) Let’s extend our gratitude to the outgoing Core Team members: Baptiste Daroussin (bapt) Benno Rice (benno) Ed Maste (emaste) George V. Neville-Neil (gnn) Matthew Seaman (matthew) Matthew, after having served as the Core Team Secretary for the past four years, will be stepping down from that role. The Core Team would also like to thank Dag-Erling Smørgrav for running a flawless election. To read about the responsibilities of the Core Team, refer to https://www.freebsd.org/administration.html#t-core. ###NetBSD WiFi refresh The NetBSD Foundation is pleased to announce a summer 2018 contract with Philip Nelson (phil%NetBSD.org@localhost) to update the IEEE 802.11 stack basing the update on the FreeBSD current code. The goals of the project are: Minimizing the differences between the FreeBSD and NetBSD IEEE 802.11 stack so future updates are easier. Adding support for the newer protocols 801.11/N and 802.11/AC. Improving SMP support in the IEEE 802.11 stack. Adding Virtual Access Point (VAP) support. Updating as many NIC drivers as time permits for the updated IEEE 802.11 stack and VAP changes. Status reports will be posted to tech-net%NetBSD.org@localhost every other week while the contract is active. iXsystems ###Poor Man’s CI - Hosted CI for BSD with shell scripting and duct tape Poor Man’s CI (PMCI - Poor Man’s Continuous Integration) is a collection of scripts that taken together work as a simple CI solution that runs on Google Cloud. While there are many advanced hosted CI systems today, and many of them are free for open source projects, none of them seem to offer a solution for the BSD operating systems (FreeBSD, NetBSD, OpenBSD, etc.) The architecture of Poor Man’s CI is system agnostic. However in the implementation provided in this repository the only supported systems are FreeBSD and NetBSD. Support for additional systems is possible. Poor Man’s CI runs on the Google Cloud. It is possible to set it up so that the service fits within the Google Cloud “Always Free” limits. In doing so the provided CI is not only hosted, but is also free! (Disclaimer: I am not affiliated with Google and do not otherwise endorse their products.) ARCHITECTURE A CI solution listens for “commit” (or more usually “push”) events, builds the associated repository at the appropriate place in its history and reports the results. Poor Man’s CI implements this very basic CI scenario using a simple architecture, which we present in this section. Poor Man’s CI consists of the following components and their interactions: Controller: Controls the overall process of accepting GitHub push events and starting builds. The Controller runs in the Cloud Functions environment and is implemented by the files in the controller source directory. It consists of the following components: Listener: Listens for GitHub push events and posts them as work messages to the workq PubSub. Dispatcher: Receives work messages from the workq PubSub and a free instance name from the Builder Pool. It instantiates a builder instance named name in the Compute Engine environment and passes it the link of a repository to build. Collector: Receives done messages from the doneq PubSub and posts the freed instance name back to the Builder Pool. PubSub Topics: workq: Transports work messages that contain the link of the repository to build. poolq: Implements the Builder Pool, which contains the name’s of available builder instances. To acquire a builder name, pull a message from the poolq. To release a builder name, post it back into the poolq. doneq: Transports done messages (builder instance terminate and delete events). These message contain the name of freed builder instances. builder: A builder is a Compute Engine instance that performs a build of a repository and shuts down when the build is complete. A builder is instantiated from a VM image and a startx (startup-exit) script. Build Logs: A Storage bucket that contains the logs of builds performed by builder instances. Logging Sink: A Logging Sink captures builder instance terminate and delete events and posts them into the doneq. BUGS The Builder Pool is currently implemented as a PubSub; messages in the PubSub contain the names of available builder instances. Unfortunately a PubSub retains its messages for a maximum of 7 days. It is therefore possible that messages will be discarded and that your PMCI deployment will suddenly find itself out of builder instances. If this happens you can reseed the Builder Pool by running the commands below. However this is a serious BUG that should be fixed. For a related discussion see https://tinyurl.com/ybkycuub. $ ./pmci queuepost poolq builder0 # ./pmci queuepost poolq builder1 # ... repeat for as many builders as you want The Dispatcher is implemented as a Retry Background Cloud Function. It accepts work messages from the workq and attempts to pull a free name from the poolq. If that fails it returns an error, which instructs the infrastructure to retry. Because the infrastructure does not provide any retry controls, this currently happens immediately and the Dispatcher spins unproductively. This is currently mitigated by a “sleep” (setTimeout), but the Cloud Functions system still counts the Function as running and charges it accordingly. While this fits within the “Always Free” limits, it is something that should eventually be fixed (perhaps by the PubSub team). For a related discussion see https://tinyurl.com/yb2vbwfd. ###The Power of Ctrl-T Did you know that you can check what a process is doing by pressing CTRL+T? Has it happened to you before that you were waiting for something to be finished that can take a lot of time, but there is no easy way to check the status. Like a dd, cp, mv and many others. All you have to do is press CTRL+T where the process is running. This will output what’s happening and will not interrupt or mess with it in any way. This causes the operating system to output the SIGINFO signal. On FreeBSD it looks like this: ping pingtest.com PING pingtest.com (5.22.149.135): 56 data bytes 64 bytes from 5.22.149.135: icmpseq=0 ttl=51 time=86.232 ms 64 bytes from 5.22.149.135: icmpseq=1 ttl=51 time=85.477 ms 64 bytes from 5.22.149.135: icmpseq=2 ttl=51 time=85.493 ms 64 bytes from 5.22.149.135: icmpseq=3 ttl=51 time=85.211 ms 64 bytes from 5.22.149.135: icmpseq=4 ttl=51 time=86.002 ms load: 1.12 cmd: ping 94371 [select] 4.70r 0.00u 0.00s 0% 2500k 5/5 packets received (100.0%) 85.211 min / 85.683 avg / 86.232 max 64 bytes from 5.22.149.135: icmpseq=5 ttl=51 time=85.725 ms 64 bytes from 5.22.149.135: icmp_seq=6 ttl=51 time=85.510 ms As you can see it not only outputs the name of the running command but the following parameters as well: 94371 – PID 4.70r – since when is the process running 0.00u – user time 0.00s – system time 0% – CPU usage 2500k – resident set size of the process or RSS `` > An even better example is with the following cp command: cp FreeBSD-11.1-RELEASE-amd64-dvd1.iso /dev/null load: 0.99 cmd: cp 94412 [runnable] 1.61r 0.00u 0.39s 3% 3100k FreeBSD-11.1-RELEASE-amd64-dvd1.iso -> /dev/null 15% load: 0.91 cmd: cp 94412 [runnable] 2.91r 0.00u 0.80s 6% 3104k FreeBSD-11.1-RELEASE-amd64-dvd1.iso -> /dev/null 32% load: 0.91 cmd: cp 94412 [runnable] 4.20r 0.00u 1.23s 9% 3104k FreeBSD-11.1-RELEASE-amd64-dvd1.iso -> /dev/null 49% load: 0.91 cmd: cp 94412 [runnable] 5.43r 0.00u 1.64s 11% 3104k FreeBSD-11.1-RELEASE-amd64-dvd1.iso -> /dev/null 64% load: 1.07 cmd: cp 94412 [runnable] 6.65r 0.00u 2.05s 13% 3104k FreeBSD-11.1-RELEASE-amd64-dvd1.iso -> /dev/null 79% load: 1.07 cmd: cp 94412 [runnable] 7.87r 0.00u 2.43s 15% 3104k FreeBSD-11.1-RELEASE-amd64-dvd1.iso -> /dev/null 95% > I prcessed CTRL+T six times. Without that, all the output would have been is the first line. > Another example how the process is changing states: wget https://download.freebsd.org/ftp/releases/amd64/amd64/ISO-IMAGES/11.1/FreeBSD-11.1-RELEASE-amd64-dvd1.iso –2018-06-17 18:47:48– https://download.freebsd.org/ftp/releases/amd64/amd64/ISO-IMAGES/11.1/FreeBSD-11.1-RELEASE-amd64-dvd1.iso Resolving download.freebsd.org (download.freebsd.org)… 96.47.72.72, 2610:1c1:1:606c::15:0 Connecting to download.freebsd.org (download.freebsd.org)|96.47.72.72|:443… connected. HTTP request sent, awaiting response… 200 OK Length: 3348465664 (3.1G) [application/octet-stream] Saving to: ‘FreeBSD-11.1-RELEASE-amd64-dvd1.iso’ FreeBSD-11.1-RELEASE-amd64-dvd1.iso 1%[> ] 41.04M 527KB/s eta 26m 49sload: 4.95 cmd: wget 10152 waiting 0.48u 0.72s FreeBSD-11.1-RELEASE-amd64-dvd1.iso 1%[> ] 49.41M 659KB/s eta 25m 29sload: 12.64 cmd: wget 10152 waiting 0.55u 0.85s FreeBSD-11.1-RELEASE-amd64-dvd1.iso 2%[=> ] 75.58M 6.31MB/s eta 20m 6s load: 11.71 cmd: wget 10152 running 0.73u 1.19s FreeBSD-11.1-RELEASE-amd64-dvd1.iso 2%[=> ] 85.63M 6.83MB/s eta 18m 58sload: 11.71 cmd: wget 10152 waiting 0.80u 1.32s FreeBSD-11.1-RELEASE-amd64-dvd1.iso 14%[==============> ] 460.23M 7.01MB/s eta 9m 0s 1 > The bad news is that CTRl+T doesn’t work with Linux kernel, but you can use it on MacOS/OS-X: —> Fetching distfiles for gmp —> Attempting to fetch gmp-6.1.2.tar.bz2 from https://distfiles.macports.org/gmp —> Verifying checksums for gmp —> Extracting gmp —> Applying patches to gmp —> Configuring gmp load: 2.81 cmd: clang 74287 running 0.31u 0.28s > PS: If I recall correctly Feld showed me CTRL+T, thank you! Beastie Bits Half billion tries for a HAMMER2 bug (http://lists.dragonflybsd.org/pipermail/commits/2018-May/672263.html) OpenBSD with various Desktops OpenBSD 6.3 running twm window manager (https://youtu.be/v6XeC5wU2s4) OpenBSD 6.3 jwm and rox desktop (https://youtu.be/jlSK2oi7CBc) OpenBSD 6.3 cwm youtube video (https://youtu.be/mgqNyrP2CPs) pf: Increase default state table size (https://svnweb.freebsd.org/base?view=revision&revision=336221) *** Tarsnap Feedback/Questions Ben Sims - Full feed? (http://dpaste.com/3XVH91T#wrap) Scott - Questions and Comments (http://dpaste.com/08P34YN#wrap) Troels - Features of FreeBSD 11.2 that deserve a mention (http://dpaste.com/3DDPEC2#wrap) Fred - Show Ideas (http://dpaste.com/296ZA0P#wrap) Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) iXsystems It's all NAS (https://www.ixsystems.com/blog/its-all-nas/)
Episode 254: Bare the OS | BSD Now 254
Control flow integrity with HardenedBSD, fixing bufferbloat with OpenBSD’s pf, Bareos Backup Server on FreeBSD, MeetBSD CfP, crypto simplified interface, twitter gems, interesting BSD commits, and more. ##Headlines Silent Fanless FreeBSD Desktop/Server Today I will write about silent fanless FreeBSD desktop or server computer … or NAS … or you name it, it can have multa##Headlines ###Cross-DSO CFI in HardenedBSD Control Flow Integrity, or CFI, raises the bar for attackers aiming to hijack control flow and execute arbitrary code. The llvm compiler toolchain, included and used by default in HardenedBSD 12-CURRENT/amd64, supports forward-edge CFI. Backward-edge CFI support is gained via a tangential feature called SafeStack. Cross-DSO CFI builds upon ASLR and PaX NOEXEC for effectiveness. HardenedBSD supports non-Cross-DSO CFI in base for 12-CURRENT/amd64 and has it enabled for a few individual ports. The term “non-Cross-DSO CFI” means that CFI is enabled for code within an application’s codebase, but not for the shared libraries it depends on. Supporting non-Cross-DSO CFI is an important initial milestone for supporting Cross-DSO CFI, or CFI applied to both shared libraries and applications. This article discusses where HardenedBSD stands with regards to Cross-DSO CFI in base. We have made a lot of progress, yet we’re not even half-way there. Brace yourself: This article is going to be full of references to “Cross-DSO CFI.” Make a drinking game out of it. Or don’t. It’s your call. ;) Using More llvm Toolchain Components CFI requires compiling source files with Link-Time Optimization (LTO). I remembered hearing a few years back that llvm developers were able to compile the entirety of FreeBSD’s source code with LTO. Compiling with LTO produces intermediate object files as LLVM IR bitcode instead of ELF objects. In March of 2017, we started compiling all applications with LTO and non-Cross-DSO CFI. This also enabled ld.lld as the default linker in base since CFI requires lld. Commit f38b51668efcd53b8146789010611a4632cafade made the switch to ld.lld as the default linker while enabling non-Cross-DSO CFI at the same time. Building libraries in base requires applications like ar, ranlib, nm, and objdump. In FreeBSD 12-CURRENT, ar and ranlib are known as “BSD ar” and “BSD ranlib.” In fact, ar and ranlib are the same applications. One is hardlinked to another and the application changes behavior depending on arvgv[0] ending in “ranlib”. The ar, nm, and objdump used in FreeBSD do not support LLVM IR bitcode object files. In preparation for Cross-DSO CFI support, commit fe4bb0104fc75c7216a6dafe2d7db0e3f5fe8257 in October 2017 saw HardenedBSD switching ar, ranlib, nm, and objdump to their respective llvm components. The llvm versions due support LLVM IR bitcode object files (surprise!) There has been some fallout in the ports tree and we’ve added LLVM_AR_UNSAFE and friends to help transition those ports that dislike llvm-ar, llvm-ranlib, llvm-nm, and llvm-objdump. With ld.lld, llvm-ar, llvm-ranlib, llvm-nm, and llvm-objdump the default, HardenedBSD has effectively switched to a full llvm compiler toolchain in 12-CURRENT/amd64. Building Libraries With LTO The primary 12-CURRENT development branch in HardenedBSD (hardened/current/master) only builds applications with LTO as mentioned in the secion above. My first attempt at building all static and shared libraries failed due to issues within llvm itself. I reported these issues to FreeBSD. Ed Maste (emaste@), Dimitry Andric (dim@), and llvm’s Rafael Espindola expertly helped address these issues. Various commits within the llvm project by Rafael fully and quickly resolved the issues brought up privately in emails. With llvm fixed, I could now build nearly every library in base with LTO. I noticed, however, that if I kept non-Cross-DSO CFI and SafeStack enabled, all applications would segfault. Even simplistic applications like /bin/ls. Disabling both non-Cross-DSO CFI and SafeStack, but keeping LTO produced a fully functioning world! I have spent the last few months figuring out why enabling either non-Cross-DSO CFI or SafeStack caused issues. This brings us to today. The Sanitizers in FreeBSD FreeBSD brought in all the files required for SafeStack and CFI. When compiling with SafeStack, llvm statically links a full sanitization framework into the application. FreeBSD includes a full copy of the sanitization framework in SafeStack, including the common C++ sanization namespaces. Thus, libclang_rt.safestack included code meant to be shared among all the sanitizers, not just SafeStack. I had naively taken a brute-force approach to setting up the libclang_rt.cfi static library. I copied the Makefile from libclang_rt.safestack and used that as a template for libclang_rt.cfi. This approach was incorrect due to breaking the One Definition Rule (ODR). Essentially, I ended up including a duplicate copy of the C++ classes and sanitizer runtime if both CFI and SafeStack were used. In my Cross-DSO CFI development VM, I now have SafeStack disabled across-the-board and am only compiling in CFI. As of 26 May 2018, an LTO-ified world (libs + apps) works in my limited testing. /bin/ls does not crash anymore! The second major milestone for Cross-DSO CFI has now been reached. Known Issues And Limitations There are a few known issues and regressions. Note that this list of known issues essentially also constitutes a “work-in-progress” and every known issue will be fixed prior to the official launch of Cross-DSO CFI. It seems llvm does not like statically compiling applications with LTO that have a mixture of C and C++ code. /sbin/devd is one of these applications. As such, when Cross-DSO CFI is enabled, devd is compiled as a Position-Independent Executable (PIE). Doing this breaks UFS systems where /usr is on a separate partition. We are currently looking into solving this issue to allow devd to be statically compiled again. NO_SHARED is now unset in the tools build stage (aka, bootstrap-tools, cross-tools). This is related to the static compilation issue above. Unsetting NO_SHARED for to tools build stage is only a band-aid until we can resolve static compliation with LTO. One goal of our Cross-DSO CFI integration work is to be able to support the cfi-icall scheme when dlopen(3) and dlsym(3)/dlfunc(3) is used. This means the runtime linker (RTLD), must be enhanced to know and care about the CFI runtime. This enhancement is not currently implemented, but is planned. When Cross-DSO CFI is enabled, SafeStack is disabled. This is because compiling with Cross-DSO CFI brings in a second copy of the sanitizer runtime, violating the One Definition Rule (ODR). Resolving this issue should be straightforward: Unify the sanitizer runtime into a single common library that both Cross-DSO CFI and SafeStack can link against. When the installed world has Cross-DSO CFI enabled, performing a buildworld with Cross-DSO CFI disabled fails. This is somewhat related to the static compilation issue described above. Current Status I’ve managed to get a Cross-DSO CFI world booting on bare metal (my development laptop) and in a VM. Some applications failed to work. Curiously, Firefox still worked (which also means xorg works). I’m now working through the known issues list, researching and learning. Future Work Fixing pretty much everything in the “Known Issues And Limitations” section. ;P I need to create a static library that includes only a single copy of the common sanitizer framework code. Applications compiled with CFI or SafeStack will then only have a single copy of the framework. Next I will need to integrate support in the RTLD for Cross-DSO CFI. Applications with the cfi-icall scheme enabled that call functions resolved through dlsym(3) currently crash due to the lack of RTLD support. I need to make a design decision as to whether to only support adding cfi-icall whitelist entries only with dlfunc(3) or to also whitelist cfi-icall entries with the more widely used dlsym(3). There’s likely more items in the “TODO” bucket that I am not currently aware of. I’m treading in uncharted territory. I have no firm ETA for any bit of this work. We may gain Cross-DSO CFI support in 2018, but it’s looking like it will be later in either 2019 or 2020. Conclusion I have been working on Cross-DSO CFI support in HardenedBSD for a little over a year now. A lot of progress is being made, yet there’s still some major hurdles to overcome. This work has already helped improve llvm and I hope more commits upstream to both FreeBSD and llvm will happen. We’re getting closer to being able to send out a preliminary Call For Testing (CFT). At the very least, I would like to solve the static linking issues prior to publishing the CFT. Expect it to be published before the end of 2018. I would like to thank Ed Maste, Dimitry Andric, and Rafael Espindola for their help, guidance, and support. iXsystems FreeNAS 11.2-BETAs are starting to appear ###Bareos Backup Server on FreeBSD Ever heard about Bareos? Probably heard about Bacula. Read what is the difference here – Why Bareos forked from Bacula? Bareos (Backup Archiving Recovery Open Sourced) is a network based open source backup solution. It is 100% open source fork of the backup project from bacula.org site. The fork is in development since late 2010 and it has a lot of new features. The source is published on github and licensed under AGPLv3 license. Bareos supports ‘Always Incremental backup which is interesting especially for users with big data. The time and network capacity consuming full backups only have to be taken once. Bareos comes with WebUI for administration tasks and restore file browser. Bareos can backup data to disk and to tape drives as well as tape libraries. It supports compression and encryption both hardware-based (like on LTO tape drives) and software-based. You can also get professional services and support from Bareos as well as Bareos subscription service that provides you access to special quality assured installation packages. I started my sysadmin job with backup system as one of the new responsibilities, so it will be like going back to the roots. As I look on the ‘backup’ market it is more and more popular – especially in cloud oriented environments – to implement various levels of protection like GOLD, SILVER and BRONZE for example. They of course have different retention times, number of backups kept, different RTO and RPO. Below is a example implementation of BRONZE level backups in Bareos. I used 3 groups of A, B and C with FULL backup starting on DAY 0 (A group), DAY 1 (B group) and DAY 2 (C group). This way you still have FULL backups quite often and with 3 groups you can balance the network load. I for the days that we will not be doing FULL backups we will be doing DIFFERENTIAL backups. People often confuse them with INCREMENTAL backups. The difference is that DIFFERENTIAL backups are always against FULL backup, so its always ‘one level of combining’. INCREMENTAL ones are done against last done backup TYPE, so its possible to have 100+ levels of combining against 99 earlier INCREMENTAL backups and the 1 FULL backup. That is why I prefer DIFFERENTIAL ones here, faster recovery. That is all backups is about generally, recovery, some people/companies tend to forget that. The implementation of BRONZE in these three groups is not perfect, but ‘does the job’. I also made ‘simulation’ how these group will overlap at the end/beginning of the month, here is the result. Not bad for my taste. Today I will show you how to install and configure Bareos Server based on FreeBSD operating system. It will be the most simplified setup with all services on single machine: bareos-dir bareos-sd bareos-webui bareos-fd I also assume that in order to provide storage space for the backup data itself You would mount resources from external NFS shares. To get in touch with Bareos terminology and technology check their great Manual in HTML or PDF version depending which format You prefer for reading documentation. Also their FAQ provides a lot of needed answers. Also this diagram may be useful for You to get some grip into the Bareos world. System As every system needs to have its name we will use latin word closest to backup here – replica – for our FreeBSD system hostname. The install would be generally the same as in the FreeBSD Desktop – Part 2 – Install article. Here is our installed FreeBSD system with login prompt.