Created by three guys who love BSD, we cover the latest news and have an extensive series of tutorials, as well as interviews with various people from all areas of the BSD community. It also serves as a platform for support and questions. We love and advocate FreeBSD, OpenBSD, NetBSD, DragonFlyBSD and TrueOS. Our show aims to be helpful and informative for new users that want to learn about them, but still be entertaining for the people who are already pros. The show airs on Wednesdays at 2:00PM (US Eastern time) and the edited version is usually up the following day.
Similar Podcasts
Thinking Elixir Podcast
The Thinking Elixir podcast is a weekly show where we talk about the Elixir programming language and the community around it. We cover news and interview guests to learn more about projects and developments in the community.
The Cynical Developer
A UK based Technology and Software Developer Podcast that helps you to improve your development knowledge and career,
through explaining the latest and greatest in development technology and providing you with what you need to succeed as a developer.
Elixir Outlaws
Elixir Outlaws is an informal discussion about interesting things happening in Elixir. Our goal is to capture the spirit of a conference hallway discussion in a podcast.
154: Myths, Pi’s & Features, oh my!
This week on BSDNow, we are taking a look at a few different tutorials, including running your very own RPi web-server. (Come-on, you This episode was brought to you by Headlines broken features aren't used (http://www.tedunangst.com/flak/post/broken-features-arent-used) This post from TedU talks about the difficulty of removing features from an operating system “One of the difficulties in removing a feature is identifying all the potential users. A feature here could be a program bundled with an operating system, or a command line option, or maybe just a function in a library. If we remove a feature, users that depend on it will be sad. Unfortunately, absence of evidence is not evidence of absence. I’ve never heard of anybody running ls -p but it’s not impossible that somebody does.” “The reasons why we want to remove an existing feature can vary. Sometimes it’s old code that interferes with maintenance. Sometimes a nearly complete rewrite can improve performance. In other cases, the feature in question is really more of a misfeature. It may have security implications, where the existence of the feature can be used to facilitate the exploitation of other vulnerabilities, and removing the feature will help mitigate the exploit.” “There’s no general test that can be used, but there is one test that works in many cases. Test that the feature works. If the feature doesn’t work, that’s compelling evidence that nobody is using it, because nobody can be using it. You don’t need to fix it. You can just remove it.” He makes some interesting comments about exhaustive unit tests and the push to keep everything working all the time. If you never break anything to see if someone complains, how do you know if it is still being used? *** A Raspberry Pi FreeBSD Web Server (http://devio.us/~spacemonkey/raspberry_pi_freebsd_web_server_howto.html) Looking at a super-low power solution to host some webpages? If so, we have the tutorial for you. Specifically a walkthrough of getting FreeBSD up on a Pi, and setting up nginx, OpenNTPD, LibreSSL and friends. The walkthrough starts with grabbing a FreeBSD 11 snapshot for arm64 and doing the initial setup process to get to a bootable FreeBSD system. If you are an extreme noob, not to fear. The tutorial walks you through setting up usernames, timezones, even a larger /tmp directory on your new MiniBSD setup. The tedious part comes to play during the setup of packages. The author walks us through setting up LibreSSL and various other packages via ports (Since LibreSSL isn’t the default in FreeBSD). This will take some time to compile on your humble RPi device. (Go make a sandwich, walk the dog, fix the gutters, etc) When it’s all said and done, you’ll end up with a secure little web-server that you’ve configured all by yourself! (Wondering what the word-press performance would be like on that box) *** Uber switches from PostgreSQL back to MySQL (https://eng.uber.com/mysql-migration/) We often hear success stories of people switching to PostgreSQL and getting huge performance gains, but this stories is the reverse Uber’s engineering team has switched back to MySQL, because for their specific workload and design, MySQL’s innodb has better performance Of course, it is not just vanilla MySQL, but “Schemaless”, a sharding system that sits on top of MySQL The article goes into detail about the on-disk format used by Postgres, and the specific shortcomings that Uber encountered Uber admits that all of its testing was against the older PostgreSQL 9.2, but one of their complaints is about having difficulty upgrading “We started out with Postgres 9.1 and successfully completed the upgrade process to move to Postgres 9.2. However, the process took so many hours that we couldn’t afford to do the process again. By the time Postgres 9.3 came out, Uber’s growth increased our dataset substantially, so the upgrade would have been even lengthier. For this reason, our legacy Postgres instances run Postgres 9.2 to this day, even though the current Postgres GA release is 9.5.” There is a followup, from the Postgres side “Why we lost Uber as a user” (https://www.postgresql.org/message-id/5797D5A1.5030009%40agliodbs.com) This thread goes into detail about the specific types of problematic queries that Uber was using “The Uber guy is right that InnoDB handles this better as long as you don't touch the primary key (primary key updates in InnoDB are really bad)” “This is a common problem case we don't have an answer for yet.” The thread then goes on to discuss possibly supporting a “pluggable heap storage layer”, to allow different workloads to use different on-disk formats for best performance *** Getting started with GhostBSD and FreeBSD Part 1 (http://unsolicitedbutoffered.blogspot.com/2016/07/getting-started-with-ghostbsd-and.html) Part 2 (http://unsolicitedbutoffered.blogspot.com/2016/07/getting-started-with-ghostbsd-and_31.html) Part 3 (http://unsolicitedbutoffered.blogspot.com/2016/08/getting-started-with-ghostbsd-and.html) Part 4 (http://unsolicitedbutoffered.blogspot.com/2016/08/getting-started-with-ghostbsd-and_2.html) In what may be our first GhostBSD tutorial, we have a nice walkthrough on the initial getting started with it. For those who don’t know, GhostBSD provides a nice XFCE or Mate desktop out of box, and still supports 32bit installs for those who want to keep that older hardware running. The walkthough takes us through the process of grabbing GhostBSD images and getting the installer up and running via bootable USB stick. Once booted, the graphical installer is straight-forward and short, allowing you to get the bits on disk as quickly as possible. (The actual installation took around 45 Minutes on an old Toshiba NB520) The author then takes us on a tour of some of GhostBSD’s out-of-box bundled applications (Along with XFCE) and how it compares to similar Linux setups. Lastly covered is the setup of Wireless (The manual way with WPA supplicant, since the GUI tool appeared to not work in this particular case) All in all a good walkthrough, especially if you’ve not seen GhostBSD in action before, the screenshots are very informative! *** News Roundup Steam on FreeBSD 11-CURRENT (https://github.com/SteamOnFreeBSD/SteamOnFreeBSD) Steam on FreeBSD. Yes, we’ve heard of setups using WINE, but what about running the Linux binaries natively? Well you are in luck. We have a github project that details getting the Linux native client up and running on a FreeBSD 11-CURRENT system. This github project is rather mysterious, with only the instructions to download a pre-packaged steam.txz file, extract and run the provided install.sh script. Curious I inspected some of the scripts, the installer.sh is fairly straight-forward, but does some ‘non-standard’ freebsd things, like fetching packages and extracting specific files/libaries into a new /compat/ubuntu directory. After that, it goes through a huge list of debian/ubuntu packages, also throwing them into the aforementioned ubuntu directory. At runtime, the wrapper script ensures that various linux compat file-systems are mounted in the correct location, then proceeds to run steam with some LDLIBRARYFLAGS set from the users .local/share/steam directory. A tad scary if honest, however it is a neat PoC to see Steam working on FreeBSD. Hopefully somebody can turn this into a more traditional package which can be easily removed / cleaned up afterwards. *** How to run Enlightenment on OpenBSD (http://enform.haxlab.org/) Are you an enlightenment fan? Be honest, we know there are quite a few of you out there! If so, we have a tutorial for you today, which talks about how to run E on OpenBSD. The process is pretty easy, but some steps might be overlooked if you are new to OpenBSD or don’t know how to tune / compile things on your own. It starts out with adjusting some sysctl’s for better tuning that works on an E based desktop. Next is installing from package some pre-reqs that will enable us to build E from source. After that, we need to download and install EFL from github, and the autogen / gmake commands are helpfully provided for you. Lastly the same is done for E itself, and TADA, E is installed and ready to go on your OpenBSD system. If you do this right, should only take 5-10 minutes to be up and running. *** Myths about FreeBSD (https://wiki.freebsd.org/Myths) Over on the FreeBSD wiki, we have a new “Myths” page which we’ve never highlighted on the show before. First up, and one I’ve very familiar with, is the usual “FreeBSD is only for Servers and not Desktops”, along with a good rebuttal about what it does offer and mention of projects such as PC-BSD which do it also. Another prevalent one is the “FreeBSD has a closed development model”, which is easily refuted: FreeBSD has over 400 developers around the world who have commit access to the repository. Many of these are willing to commit patches from third parties. If you want to get an idea of the number of patches that have been committed on behalf of other developers, then search for 'Submitted by' in the commit logs. At the time of writing, this is just under twenty thousand, or about ten percent of all commits. After having a few patches accepted, regular contributors are usually encouraged to apply for commit access. Another one that we are still hearing (Although it is less and less now) was the common “FreeBSD makes me compile everything from source”. Listeners of this show will know that pkg has pretty much made this irrelevant in recent years. However the option to compile yourself from source still exists, but most users won’t find this ever necessary. A good list, with many more items on it than we’ve mentioned here. Take a look, you might find something there you’ve heard in the wild, or maybe even thought yourself at one point! FreeBSD Area51 testing repo, KDE 5 (https://gist.github.com/swills/8a9a9f8a529256f134830c2a1476db0c) This github GIST from Steve Wills provides the instructions to enable the FreeBSD xorg teams unofficial ‘Area51’ testing repo on your machine This gives you access to the new KDE 5 Users should obviously be careful testing early-access software, but bug reports are very welcome, and important to getting KDE 5 working well under FreeBSD *** Beastie Bits NetBSD Machines at Open Source Conference 2016 Kyoto (http://mail-index.netbsd.org/netbsd-advocacy/2016/08/01/msg000712.html) How to install htop on pfSense (http://www.cyberciti.biz/faq/pfsense-install-htop-using-pkg-command/) The first ever patch for #Chromium on #NetBSD has been committed! (https://twitter.com/krytarowski/status/759909139300491265) CharmBug - Adventures in Hardened BSD - August 24th 2016 (http://www.meetup.com/CharmBUG/events/232474857/) Feedback/Questions Emanual - GoTTY (http://pastebin.com/q24sNcNY) Joe - Disable Device (http://pastebin.com/iTkmpVr9) Allen - ZFS Send/Recv (http://pastebin.com/bPHEP4Ya) Mica - Adobe Replacements (http://pastebin.com/L0ttuNx3) Robroy - RAID (http://pastebin.com/Dr92CGCU) ***
153: Big int trouble
This week on BSDNow, we have a variety of news to discuss, covering quite the spectrum of BSD. (Including a new DragonFly release!). This episode was brought to you by Headlines my int is too big (http://www.tedunangst.com/flak/post/my-int-is-too-big) “The NCC Group report (http://marc.info/?l=oss-security&m=146853062403622&w=2) describes the bugs, but not the history of the code.” “Several of them, as reported by NCC, involved similar integer truncation issues. Actually, they involved very similar modern 64 bit code meeting classic 32 bit code” “The thrsleep system call is a part of the kernel code that supports threads. As the name implies, it gives userland a measure of control over scheduling and lets a thread sleep until something happens. As such, it takes a timeout in the form of a timespec. The kernel, however, internally implements time keeping using ticks (there are HZ, 100, ticks per second). The tsleep function (t is for timed) takes an int number of ticks and performs basic validation by checking that it’s not negative. A negative timeout would indicate that the caller has miscalculated. The kernel panics so you can fix the bug, instead of stalling forever.” “The trouble therefore is when userland is allowed to specify a timeout that could be negative. The existing code made an attempt to handle various tricks by converting the timespec to a ticks value stored as a 64 bit long long which was checked against INTMAX before passing to sleep. Any value over INTMAX would be truncated, so we can’t allow that. Instead, we saturate the value to INT_MAX. Unfortunately, this check didn’t account for the possibility that the tick conversion from the timespec could also overflow and result in a negative value.” Then there is the description of the kqueue flaw: “Every kqueue keeps a list of all the attached events it’s watching for. A simple array is used to store file events, indexed by fd.” “This array is scaled to accommodate the largest fd that needs to be stored. This would obviously cause trouble, consuming too much memory, if the identifier were not validated first. Which is exactly what kqueue tries to do. The fdgetfile function checks that the identifier is a file that the process has open. One wrinkle. fdgetfile takes an int argument but ident is a uintptr_t, possibly 64 bits. An ident of 2^32 + 2 will look like a valid file descriptor, but then cause the array to be resized to gargantuan proportions.” “Again, the fix is pretty simple. We must check that the ident is bounded by INTMAX before calling fdgetfile. This bug likely would have been exploitable beyond a panic, but the array allocation was changed to use mallocarray instead of multiplying arguments by hand, thus preventing another overflow.” Then there is a description of the anonymous mmap flaw, and the “secret magic” _MAPNOFAULT flag *** FreeBSD Quarterly Status Report Q2 2016 (https://www.freebsd.org/news/status/report-2016-04-2016-06.html) It’s time for another round of FreeBSD Quarterly Status Reports! In this edition, we have status updates from the various teams, including IRC/Bugs/RE/Ports/Core and Foundation We also have updates on some specific projects, including from Konstantin on the on-going work for his implementation of ASLR, including the new ‘proccontrol’ command which provides the following: > “The proccontrol(1) utility was written to manage and query ASLR enforcement on a per-process basis. It is required for analyzing ASLR failures in specific programs. This utility leverages the procctl(2) interface which was added to the previous version of the patch, with some bug fixes.” Next are updates on porting CEPH to FreeBSD, the ongoing work to improve EFI+GELI (touched on last week) and more robust Mutexes. Additionally we have an update from Matt Macy and the Xorg team discussing the current work to update FreeBSD’s graphic stack: > “All Intel GPUs up to and including the unreleased Kaby Lake are supported. The xf86-video-intel driver will be updated soon. Updating this driver requires updating Xorg, which in turn is blocked on Nvidia updates.” The kernel also got some feature status updates, including on the new Allwinner SoC support, an update on FreeBSD in Hyper-V and VIMAGE In addition to a quick update on the arm64 architecture (It’s getting there, RPi3 is almost a thing), we also have a slew of port updates, including support for GitLab in ports, updates on GNOME / KDE and some additional Intel-specific networking tools. *** Vulnerabilities discovered in freebsd-update and portsnap (https://lists.freebsd.org/pipermail/freebsd-security/2016-July/009016.html) There are two vulnerabilities discovered in freebsd-update and portsnap, where an attacker could place files in the portsnap directory and they would be used without being subject to having their checksum verified (but this requires root access), and the second where a man-in-the-middle attacker could guess the name of a file you will fetch by exploiting the time-gap between when you download the initial snapshot, and when you fetch the updated files. There are a number of vulnerabilities that were discovered in libarchive/tar as well There is also an issue with bspatch. A security advisory for bspatch has already been released, as this vulnerabilities was also discovered by the Chromium team, which uses this same code. The patch discussed in this mailing list thread is larger, but secteam@ believes at least one of the additional checks introduced is incorrect and may prevent a valid patch from being applied. The smaller patch was pushed out first, to solve the main attack vector, while the larger patch is investigated. Automated fuzz testing is underway. Great care is being taken fixing bspatch, as if it is broken installing future updates becomes much more difficult secteam@ and core@ would like to emphasize that the FreeBSD project takes these issue very seriously and are working on it > “As a general rule, secteam@ does not announce vulnerabilities for which we don't have patches, but we concede that we should have considered making an exception in this case” Work is underway to re-architect freebsd-update and portsnap to do signature verification on all files before they are passed to libarchive/tar, to help protect users from any future vulnerabilities in libarchive. However, this requires changes to the metadata format to provide these additional signatures, and backwards compatibilities must be preserved, so people can update to the newer versions to get these additional security features There is also discussion of using HTTPS for delivery of the files, but certificate verification and trust are always an issue. FreeBSD does not distribute a certificate trust store by default. There will be more on this in the coming days. *** OpenSSH 7.3 Released (http://www.openssh.com/txt/release-7.3) OpenSSH 7.3 has landed! Primarily a bug-fix release, the release notes do mention the pending deprecation of some more legacy Crypto in the future, including denying all RSA keys < 1024bit, and removal of SSHv1 support. (Already disabled via compile option) On the bug side, there was a security issue addressed in sshd: “sshd(8): Mitigate a potential denial-of-service attack against the system's crypt(3) function via sshd(8). An attacker could send very long passwords that would cause excessive CPU use in crypt(3). sshd(8) now refuses to accept password authentication requests of length greater than 1024 characters” Also a timing issue was resolved in regard to password auth, which could possibly allow an attacker to discern between valid/invalid account names. On the feature side, we have the new ProxyJump option (-J flag) which allows you to do simplified indirection through various SSH jump hosts. Various bugs were fixed, and some compile failures resolved in the portable version to auto-disable some ciphers not supported by OpenSSL. News Roundup OpenBSD Ports - Integrating Third Party Applications [pdf] (http://jggimi.homeip.net/semibug.pdf) A talk from Josh Grosse, presented at SEMIBUG (South-East Michigan BSD Users Group), about OpenBSD Ports It opens by explaining the separation of the ‘base system’ from ‘packages’, as is common in most all BSDs It explains the contents of OpenBSD package tar file, which contain some metadata files (+CONTENTS and +DESC) and then the actual package files The talk goes on to explain the different branches (-release, -stable, and -current), and warn users that there are no official -stable packages from the project Then it goes on into the development model, including what new contributors should expect Then it walks through the entire process of creating a port and getting it contributed *** NetBSD removes last RWX page in amd64 kernel (http://mail-index.netbsd.org/source-changes/2016/07/27/msg076413.html) NetBSD has purged the last holdout RWX page on the amd64 platform > “Use UVMPROTALL only if UVMKMFEXEC is given as argument. Otherwise, if UVMKMFPAGEABLE is also given as argument, only the VA is allocated and UVM waits for the page to fault before kentering it. When kentering it, it will use the UVMPROT flag that was passed to uvm_map; which means that it will kenter it as RWX. With this change, the number of RWX pages in the amd64 kernel reaches strictly zero.” Break out the party favors! Hopefully any last stragglers in any of the other BSD’s gets retired soon as well. *** DragonFly BSD 4.6 launches with home-grown support for NVMe Controllers (http://linux.softpedia.com/blog/dragonfly-bsd-4-6-0-launches-with-home-grown-support-for-nvme-controllers-506908.shtml) Softpedia picked up on the release of DragonFlyBSD 4.6, specifically about their new home-grown NVMe driver. > “We now have a NVMe driver (PCIe SSDs). It currently must be kldloaded with nvme_load="YES" in /boot/loader.conf. The driver uses all concurrency features offered by the chip and will distribute queues and interrupts across multiple CPUs to maximize performance. It has been tested up to around 1.05M IOPS @4K, and roughly 6.5 GBytes/sec @32K (random read from urandom-filled partition, physio, many threads), with the 2xE5-2620v4 (xeon) test server 78% idle in the IOPS test and 72% idle on the bandwidth test. In other words, we maxed out the three NVMe devices we had plugged in and the system still had plenty of suds left over. Please note that a machine's ability to boot from an NVMe device depends on the BIOS, and not DragonFly. Most BIOSes cannot boot from NVMe devices and those that can probably only do it through UEFI. Info on device state is available with the new utility nvmectl.“ In addition to this improved support, 4.6 also brings in the improved graphics support, matching what is in Linux 4.4 and support for Broadwell/Skylake. SMP also got some love: > “SMP performance was already very good. As part of the NVMe driver work we revamped the buffer cache subsystem and a number of other I/O related paths, further reducing lock contention and IPI signalling overheads. We also put topology-aware cpu cache localization into the kernel memory allocator (primarily helps multi-socket systems and systems with high core counts). The network subsystem also continues to receive significant improvement, with modest machine configurations now capable of handling upwards of 580K conns/sec.“ +Full Release Notes (https://www.dragonflybsd.org/release46/) *** The powerd++ daemon monitors the system load and adjusts the CPU clock accordingly and is a drop-in replacement for FreeBSD's native powerd(8). (http://www.freshports.org/sysutils/powerdxx/) As mentioned in our EuroBSDCon 2016 rundown, Dominic Fandrey will be giving a presentation about his powerd replacement, powerd++ The source code is already available on github, and is in ports The major difference is the newer design handle many-core systems much better. The original powerd was written at a time when most laptops only had a single core, and maybe a hyperthread. The new design decides which CPU frequency to use by looking at the busiest core, rather than the average across the cores, resulting in a more meaningful result. It also supports averaging over a longer period of time, to avoid jumping to a higher frequency to quickly powerd++ also avoids ‘slewing’ the cpu frequency, ratching it up and down one step at a time, and instead jumps directly to the target frequency. Often times, you will use less battery by jumping to maximum frequency, finishing the work, and going back to a low power state, than trying to do that work over a longer period of time in low power mode *** Beastie Bits Hyper-V: Unmapped I/O improves userland direct disk performance by 35% ~ 135% (https://svnweb.freebsd.org/base?view=revision&revision=303474) One does not simply remove FreeBSD (https://imgur.com/a/gjGoq) A new BSD Podcast "BSD Synergy" has started (https://www.youtube.com/channel/UCBua6yMtJ6W5ExYSREnS3UQ) KnoxBug - Next Meeting - Aug 30th (http://knoxbug.org/content/2016-08-30) Feedback/Questions Daniel - Root/Wheel (http://pastebin.com/8sMyKm6c) Joe - IPV6 Frag (http://pastebin.com/r5Y0gbxf) Paul - ChicagoBug (http://pastebin.com/iVYPYcVs) Chris - SSH BruteBlock (http://pastebin.com/597m9gHa) Todd - Jails (http://pastebin.com/xjbKwSaz) ***
152: The Laporte has landed!
This week on BSDNow, we have some big breaking news about another major switcher to FreeBSD, plus early information about the pending This episode was brought to you by Headlines Leo Laporte tries FreeBSD (http://www.leolaporte.com/blog/a-grand-experiment) Leo Laporte, formerly of TechTV, and now of TWiT.tv, is switching to FreeBSD “The latest debacle over the "forced" upgrade to Windows 10 and Apple's increasingly locked-in ecosystem has got me thinking. Do I really need to use a proprietary operating system to get work done? And while I'm at it, do I need to use commercial cloud services to store my data?” A sometimes Linux user since the mid 90s, Leo talks about his motivations: “But as time went by, even Ubuntu began to seem too commercial to me” “So now for the grand experiment. Is it possible, I wonder, to do everything I need to do on an even more venerable, more robust system: a true UNIX OS, FreeBSD? Here are my requirements” Browsing Email with PGP signing and encryption Coding - I'm a hobbyist programmer requiring support for lisp/scheme/racket, rust, and python (and maybe forth and clojure and meteor and whatever else is cool and new) Writing A password vault. I currently use Lastpass because it syncs with mobile but eventually I'll need to find a FOSS replacement for that, too Photo editing - this is the toughest to replace. I love Photoshop and Lightroom. Can I get by with, say, GIMP and Darktable? I do all of those things on my PCBSD machine all the time “I love Linux and will continue to use it on my laptops, but for my main workhorse desktop I think FreeBSD will be a better choice. I also look forward to learning and administering a true UNIX system.” He got a nice SuperMicro based workstation, with an Intel Xeon E3-1275v5 and an NVIDIA GeForce GTX 960 GPU I have a server with one of those Skylake E3s, it is very nice “450Mbps Wireless N Dual Band PCI-e Adapter w/ 3x 2dBi Antennas (Yes, sad to say, unless I rewire my house I'll have to use Wi-Fi with this beast. I'll probably rewire my house.)” He plans to have a 4x 1TB ZFS pool, plus a second pool backed by a 512 GB NVMe m.2 for the OS “And I'll continue to chronicle my journey into the land of FOSS here when The Beast arrives. But in the meantime, please excuse me, I've got some reading to do.” Leo went so far as to slap a “Power By FreeBSD” sticker (https://youtu.be/vNVst_rxxm0?t=270) on the back of his new Tesla *** OpenBSD 6.0 to be released on Sept 1st, 2016 (http://undeadly.org/cgi?action=article&sid=20160725100831) OpenBSD 6.0 Tenative Released Notes (https://www.openbsd.org/60.html) OpenBSD 6.0 is just around the corner, currently slated for Sept 1st and brings with it a whole slew of exciting new features First up, and let’s get this right out of the way.. VAX support has been dropped!! Oh no! However to make up for this devastating loss, armv7 has been added to this release. The tentative release notes are very complete and marks 6.0 as quite an exciting release OpenBSD 6.0 Pre-orders up (http://undeadly.org/cgi?action=article&sid=20160726230851) OpenBSD 6.0 tightens security by losing Linux compatibility (http://www.infoworld.com/article/3099038/open-source-tools/openbsd-60-tightens-security-by-losing-linux-compatibility.html) In related news, infoworld picked up on the pending removal of Linux compat from OpenBSD 6.0. Touted as a security feature, you will soon be unable to run legacy linux binaries on OpenBSD. This has both positives and negatives depending upon your use case. Ironically we’re excitedly awaiting improved Linux Compat support in FreeBSD, to allow running some various closed-source applications. (Netflix DRM, Steam, Skype to name a few) *** EuroBSDCon 2016 Schedule released (https://2016.eurobsdcon.org/talks-schedule/) EuroBSDCon 2016 Tutorial Schedule released (https://2016.eurobsdcon.org/tutorials/) EuroBSDCon has announced the list of talks and tutorials for September 22nd-25th’s conference! George Neville Neil (Who we’ve interviewed in the past) is giving the keynote about “The Coming Decades of BSD” *** News Roundup Blast from the past No interview again this week, we’re working on getting some people lined up. The Leo Laporte story brought these old gem from TechTV into my youtube playlist: Matt Olander and Murrey Stokey explain FreeBSD on TechTV (https://www.youtube.com/watch?v=d0UsXwRvaIg) Matt Olander and Brooks Davis explain building a cluster with FreeBSD on TechTV (https://www.youtube.com/watch?v=bAsYz5pVwyc) FreeBSD vs Linux Part 1 (https://www.youtube.com/watch?v=91igg2UX7o8) FreeBSD vs Linux Part 2 (https://www.youtube.com/watch?v=oU88fQkwfws) *** Running FreeBSD on the LibreM (https://ericmccorkleblog.wordpress.com/2016/07/16/freebsd-librem-update/) Eric McCorkle (Who has worked on the EFI loader for a while now) has written an update on his efforts to get FreeBSD working properly on the LibreM 13 laptop. Since April the work seems to be progressing nicely Matt Macy’s i915 graphics patch works well on the Librem 13, and I personally made sure that the suspend/resume support works. The patch is very stable on the Librem, and I’ve only had one kernel panic the entire time testing it. The HDMI output Just Works™ with the i915 driver. Even better, it works for both X11 and console modes. Full support for the Atheros 9462 card has been merged in. I’ve had some occasional issues, but it works for the most part. The vesa weirdness is obviated by i915 support, but it was resolved by using the scfb driver. Some of the outstanding issues still being worked on are support for Synaptics on this particular touchpad, as well as hotkey support for the keyboard, and brightness controls. In addition Eric is still working on the EFI + Geli support, with the eventual goal of getting EFI secure-boot working out of box as well. More OpenBSD syscall fuzzing (http://seclists.org/oss-sec/2016/q3/157) NCC Group’s Project Triforce continues its work of fuzzing OpenBSD This time they have found a flaw that allows any user to panic the kernel Attempting to read from the tmpfs_vfsops sysctl tree will panic the system: “attempt to execute user address 0x0 in supervisor mode” This is actually a “good” thing… “Impact: Any user can panic the kernel by using the sysctl call. If a user can manage to map a page at address zero, they may be able to gain kernel code execution and escalate privileges” OpenBSD’s default configuration prevents mapping a page at address zero, so the code execution is prevented So while a panic is a bad outcome, it is a lot better than it could have been *** Root privilege escalation on NetBSD (http://akat1.pl/?id=2) This post described a root privilege escalation in NetBSD mail.local is a utility included in the base system for delivering mail to other users on the same system, rather than invoking a mail client and going through the mail server. The mail.local utility contains a ‘time of check / time of use’ vulnerability. This means that it checks if a file or permission is valid, and then later accesses that file. If an attacker can change that file between the time when it is checked, and the time when it is used, they may be able to exploit the system by evading the check This is exactly what happens in this case mail.local appends a message to the indicated user’s mailbox It first checks if the target user already has an existing mailbox file. If the file exists, but is a link, mail.local exits with an error (to prevent exploits) If the file does not exist, it is created The message is then appended to the file If the file needed to be created, it is chown’d to the owner of the mailbox This is where the problem lies, if mail.local checks and does not find the mailbox, but an attacker then creates a link from the target mailbox to some other file mail.local then appends to that file instead, thinking it is creating the new mailbox Then, mail.local chown’s the target file to the user the attacker was trying to send mail to The article explains how this could be used to replace /etc/master.passwd etc, but opts for an easier proof of concept, replacing /usr/bin/atrun, which is run as root every 5 minutes from crontab with a script that will copy the shell to /tmp and mark it setuid The attacker can then run that shell out of /tmp, and be root NetBSD fixed the vulnerability by changing the code flow, separating the cases for opening an existing file from creating a new file. In the case where an existing file is opened, the code then verifies that the file that was opened has the same inode number and is on the same device, as the file that was checked earlier, to ensure it was not a link *** FreeBSD Heap vulnerability in bspatch (https://www.freebsd.org/security/advisories/FreeBSD-SA-16:25.bspatch.asc) An important vuln has been found and fixed in FreeBSD this past week, specifically relating to the ‘bspatch’ utility. “Upstream's bspatch.c implementation doesn't check for negative values on the number of bytes to read from the "diff" and "extra" streams, allowing an attacker controlling the patch file to write at arbitrary locations in the heap.” This could result in a crash, or running arbitrary code as the user running bspatch. (Often root) “bspatch's main loop reads three numbers from the "control" stream in the patch: X, Y and Z. The first two are the number of bytes to read from "diff" and "extra" (and thus only non-negative), while the third one could be positive or negative and moves the oldpos pointer on the source image. These 3 values are 64bits signed ints (encoded somehow on the file) that are later passed the function that reads from the streams, but those values are not verified to be non-negative.” “Chrome[OS] has four different implementations of this program, all derived from the same original code by Colin Percival.” Chromium Issue Tracker (https://bugs.chromium.org/p/chromium/issues/detail?id=372525) Patch your systems now! *** Beastie Bits: If you're a BUG member or Organizer, please contact BSD Now (https://twitter.com/q5sys/status/758087886927388673) TedU writes about some interesting localizations to gcc in openbsd, and why they are there (http://www.tedunangst.com/flak/post/one-reason-to-hate-openbsd) List of Products based on FreeBSD -- Help complete the list (https://en.wikipedia.org/wiki/List_of_products_based_on_FreeBSD) Virtualbox v5 hits the FreeBSD Ports tree (http://www.freshports.org/emulators/virtualbox-ose/) Skull Canyon NUC booting FreeBSD 11.0-BETA2 (https://gist.github.com/gonzopancho/b71be467f45594822131f4816d6cb718) 2016 BSDCan Trip Report : Trent Thompson (https://www.freebsdfoundation.org/blog/2016-bsdcan-trip-report-trent-thompson/) August London BSD Meetup (http://mail-index.netbsd.org/regional-london/2016/07/25/msg000542.html) Feedback/Questions Michael Open-Source Alts (http://pastebin.com/eiWbDXTd) Herminio - AP Troubles (http://pastebin.com/w9aCDBut) Jake - Plasma (http://pastebin.com/d15QpVFw) Morgan - Clean DO Droplets (http://pastebin.com/Wj1P7jq8) Chris - Auditd (http://pastebin.com/U9PYEH6K) ***
151: Fuzzy Auditing
This week on BSDNow, we have all sorts of interesting news, including a Kernel Fuzzing audit done for OpenBSD, a much improved This episode was brought to you by Headlines Multiple Bugs in OpenBSD Kernel (http://marc.info/?l=oss-security&m=146853062403622&w=2) Its patch Wednesday! (OR last Thursday if you were watching the mailing lists) Jesse Hertz and Tim Newsham (part of the NCC Group calling themselves project Triforce) have been working with the OpenBSD team to fix some newly discovered bugs in the kernel using fuzzing. Specifically they were able to track down several potential methods to corrupt memory or panic the kernel: mmap_panic: Malicious calls to mmap() can trigger an allocation panic or trigger memory corruption. kevent_panic: Any user can panic the kernel with the kevent system call. thrsleeppanic: Any user can panic the kernel with the _thrsleep system Call. thrsigdivertpanic: Any user can panic the kernel with the _thrsigdivert system call. ufsgetdentspanic: Any user can panic the kernel with the getdents system call. mount_panic: Root users, or users on systems with kern.usermount set to true, can trigger a kernel panic when mounting a tmpfs filesystem. unmount_panic: Root users, or users on systems with kern.usermount set to true, can trigger a kernel panic when unmounting a filesystem. tmpfsmknodpanic: Root can panic kernel with mknod on a tmpfs filesystem. This was a great find, and we have a link to more of the results, if you would like to explore them in more detail. NCC Group OpenBSD Kernel fuzzing results (http://www.openwall.com/lists/oss-security/2016/07/14/5) Would like to see more work like this done in all of the BSDs *** Running CockroachDB in a FreeBSD Jail (https://www.cockroachlabs.com/blog/critters-in-a-jar-running-cockroachdb-in-a-freebsd-jail/) The developers behind CockroachDB have written up a nice walkthrough of getting their software to run inside FreeBSD jails. “Manually encapsulating CockroachDB using Linux cgroups is no easy task, which is why tools like Docker exist in the first place. By comparison, running server processes natively in FreeBSD jails is straightforward and robust.” The walkthrough begins with compiling CockroachDB straight from source (A port is pending), which is pretty easy relying upon bash / git / gmake and GO. With the compile finished, the next step will be mounting linprocfs, although that may be going away in the future: “(Note: Linux compatibility files / packages / libraries are not needed further. CockroachDB uses Linux’s procfs to inspect system properties via gosigar. If/when gosigar evolves to read FreeBSD properties natively, CockroachDB will not need linprocfs any more.)” With the initial setup complete, the walkthrough then takes us through the process of creating the rc.d script (Which should be included with the port) and ultimately setting up ezjail and deploying CockroachDB within. With the word getting out about jails and their functionality, we hope to see more projects also provide walkthroughs and FreeBSD support natively. Kudos to the CockroachDB team! *** Usermount bugs (https://marc.info/?l=openbsd-announce&m=146854517406640&w=2) kern.usermount, (vfs.usermount on FreeBSD) is a sysctl that can be enabled to allow an unprivileged user to mount filesystems. It is very useful for allowing non-root users to mount a USB stick or other external media. It is not without its dangers though: > “kern.usermount=1 is unsafe for everyone, since it allows any non-pledged program to call the mount/umount system calls. There is no way any user can be expected to keep their system safe / reliable with this feature. Ignore setting to =1, and after release we'll delete the sysctl entirely.” In OpenBSD 6.0 and forward, the setting will no longer work, and root privileges will be required to mount a filesystem If there is a bug in the filesystem driver, the user could potentially exploit that and root the system > “In addition to the patched bugs, several panics were discovered by NCC that can be triggered by root or users with the usermount option set. These bugs are not getting patched because we believe they are only the tip of the iceberg. The mount system call exposes too much code to userland to be considered secure” This is a very pragmatic way of dealing with these issues, as it is not really possible to be sure that EVERY bug has been fixed, and that this feature is no longer an exploit vector usermount being removed from OpenBSD (http://undeadly.org/cgi?action=article&sid=20160715125022) I use this facility in FreeBSD extensively, combined with ZFS permission delegation, to allow non-root users to create and mount new ZFS datasets, and to do replication without requiring any root access There are some safety belts, for instance: the user must own the directory that the new filesystem will be mounted to, so they can’t mount to /etc and replace the password file with their own *** Let's Encrypt client from BSD in C (https://kristaps.bsd.lv/letskencrypt/) File this one under the category of “It’s about time!”, but Kristaps (Who we’ve interviewed in the past) has released some new software for interacting with letsencrypt. The header for the project site sums it up nicely: “Be up-front about security: OpenSSL is known to have issues, you can't trust what comes down the pipe, and your private key's integrity is a hard requirement. Not a situation where you can be careless. letskencrypt is a client for Let's Encrypt users, but one designed for security. No Python. No Ruby. No Bash.A straightforward, open source implementation in C that isolates each step of the sequence.” What specifically does it isolate you ask? Right now it is broken down into 6 steps: read and parse an account and domain private key authenticate with the Let's Encrypt server authorise each domain listed for the certificate submit the X509 request receive and serialise the signed X509 certificate request, receive, and serialise the certificate chain from the issuer I don’t know about all of you, but I’m going to be switching over one of my systems this weekend. *** News Roundup Videos from the FOSDEM BSD Dev room are now online (https://video.fosdem.org/2016/k4601/) The videos from the BSD Dev room at FOSDEM have been stealthily posted online at some point since last I checked The videos are individually linked from the talks on the Schedule (https://archive.fosdem.org/2016/schedule/track/bsd/) The talk pages also include the slides, which can help you to follow along *** FreeBSD on Jetson TK1 (http://kernelnomicon.org/?p=628) The nVidia Jetson TK1 is a medium sized ARM device that is a big more than your standard Raspberry Pi The device has: NVIDIA 4-Plus-1™ Quad-Core ARM® Cortex™-A15 CPU (2.3 GHz) NVIDIA Kepler GPU with 192 CUDA Cores 2 GB DDR3L x16 Memory with 64-bit Width 16 GB 4.51 eMMC Memory 1 Half Mini-PCIE Slot 1 Full-Size SD/MMC Connector 1 Full-Size HDMI Port 1 USB 2.0 Port, Micro AB 1 USB 3.0 Port, A 1 RS232 Serial Port 1 ALC5639 Realtek Audio Codec with Mic In and Line Out 1 RTL8111GS Realtek GigE LAN 1 SATA Data Port SPI 4 MByte Boot Flash The following signals are available through an expansion port: DP/LVDS Touch SPI 1x4 + 1x1 CSI-2 GPIOs UART HSIC i2c The device costs $192 USD from nVidia or Amazon Oleksandr Tymoshenko (gonzo@freebsd.org) has a post describing what it takes to get FreeBSD running on the Jetson TK1 > “First of all – my TK1 didn’t have U-Boot. Type of bootloader depends on the version of Linux4Tegra TK1 comes with. Mine had L4T R19, with some kind of “not u-boot” bootloader.” They tried using the provided tool, compiled on FreeBSD since it uses libusb, but it gave an error. Falling back to trying from Ubuntu, they got the same error. They then flashed the TK1 with newer firmware, and suddenly, uboot is available. The post then walks through pxe booting FreeBSD on the TK1 The guide then walks through replacing the UBoot with a version compatible with UBLDR, for more features We’ll have to wait for another post to get FreeBSD burned onto the device, but at this point, you can reliably boot it without any user interaction I have one of these devices, so I am very interested in this work *** Why we use OpenBSD at VidiGaurd (https://blog.vidiguard.com/why-we-use-openbsd-at-vidiguard-4521f217b2b7#.9r86v742v) VidiGuard (Which makes autonomous drone solutions for security monitoring) has posted an interesting write-up on why they use OpenBSD. Specifically they start by mentioning while they are in business to provide physical security, they just as equally value their data security, especially their customer data. They name 4 specific features that matter to them, starting with Uncompromising Quality and Security: “Over the past 20 years, OpenBSD’s focus on uncompromising quality and code correctness has yielded an operating system second-to-none. Code auditing and review is core to the project’s development process. The team’s focus on security includes integrated cryptography, new security mitigation techniques, and an optional-security-is-no-security stance, making it arguably the most secure operating system available today. This approach pays off in the form of only a few security updates for a given release, compared to other operating systems that might release a handful of updates every week.” High praise indeed! They also mention the sane-defaults, documentation and last but not least, the license as also winning factors in making OpenBSD their operating system of choice. Thanks to VidiGuard for publically detailing the use of BSD, and we hope to see other business follow suit! *** "You can (and should) slow down and learn how things work" – Interview with Dru Lavigne (https://bsdmag.org/dru_lavigne/) If you’ve been around the BSD community for any length of time, you no doubt have heard of Dru Lavigne (Or perhaps own one of her books!) She was recently interviewed by Luca Ferrari for BSD Magazine and you may find it a fascinating read. The 2nd question asked sounded a lot like our opener to an interview (How did you get into BSD) “ In the mid 90s, I went back to school to learn network and system administration. As graduation grew near and I started looking for a work, I noticed that all the interesting jobs wanted Unix skills. Wanting to increase my skills, and not having any money, I did an Internet search for “Free Unix”. The first hit was freebsd.org. I went to the website and started reading the Handbook and thought “I can do this”. Since I only had access to one computer and wanted to ramp up my skills quickly, I printed out the installation and networking chapters of the Handbook. I replaced the current operating system with FreeBSD and forced myself to learn how to do everything I needed to do on that computer in FreeBSD. It was a painful (and scary) few weeks as I figured out how to transition the family’s workflow to FreeBSD, but it was also exhilarating to learn that “yes, I can do this!. Since then, I’ve had the opportunity to try out or administer the other BSDs, several Linux distros, SCO, and Solaris. I found that the layout, logic, and release engineering process of the BSDs makes the most sense to me and I’m happiest when on a BSD system.” When asked, Dru also had a good response to what challenges potential new UNIX or BSD users may face: “Students who haven’t been exposed to open source before are used to thinking of technology in terms of a purchasable brand consisting of “black boxes” that are supposed to “just work”, without having to think about how they work. You can (and should) slow down and learn how things work. It can be a mind shift to learn that the freedom to use and change how something works does exist, and isn’t considered stealing. And that learning how something works, while hard, can be fun. BSD culture, in particular, is well suited for those who have the time and temperament to dive into how things work. With over 40 years of freely available source and commit messages, you can dive as deep as you want into learning how things came to be, how they evolved over the years, how they work now, and how they can be improved. There is a diverse range of stuff to choose from: from user tools to networking to memory management to hardware drivers to security mechanisms and so on. There is also a culture of sharing and learning and encouragement for users who demonstrate that they have done their homework and have their own ideas to contribute.” The interview is quite long, and Dru provides fantastic insights into more aspects of BSD in general. Well worth your time to read! *** Beastie Bits: Ed Maste is seeking testing 'withoutgpldtc' (https://twitter.com/ed_maste/status/755474764479672321) “PAM Mastery” tech reviewers wanted (http://blather.michaelwlucas.com/archives/2717) OPNsense 16.7 RC2 (https://opnsense.org/opnsense-16-7-rc2-released/) Jupyter Notebook for bootstrapping Arduino on FreeBSD (https://nbviewer.jupyter.org/github/DadAtH-me/Projects/blob/master/arduino-on-nix.ipynb) The Design and Implementation of the Anykernel and Rump Kernels (second edition) (http://www.fixup.fi/misc/rumpkernel-book/) Complete desktop synchronisation with Unison and FreeBSD jails (xjails) (https://github.com/kbs1/freebsd-synced-xjails) Feedback/Questions Eric - List most popular files (http://pastebin.com/S7u0VeVi) Robroy - ZFS Write Cache (http://pastebin.com/81Zmj0cX) Luis - FreeNAS HW Setup (http://pastebin.com/SfeKR7v2) Emett - Python Followup (http://pastebin.com/wy4ar0YH) Peter - Multicast + Jails (http://pastebin.com/zd2QAu25) ***
150: Sprinkle a little BSD into your life.
Today on the show, we are going to be talking to Jim Brown (of BSD Cert Fame) about his home-brew sprinkler system… Wait for it… This episode was brought to you by Headlines Distrowatch reviews OpenBSD and PCBSD's live upgrade method (http://distrowatch.com/weekly.php?issue=20160620#upgrade) Upgrading… The bane of any sysadmin! Distrowatch has recently done a write-up on the in-place upgrading of various distros / BSDs including PC-BSD and OpenBSD. Lets look first at the PC-BSD attempt, which was done going from 9.2 -> 10. “I soon found trying to upgrade either the base system or pkg would fail. The update manager did not provide details as to what had gone wrong and so I decided to attempt a manual upgrade by following the FreeBSD Handbook as I had when performing a live upgrade of FreeBSD back in May. At first the manual process seemed to work, downloading the necessary patches for FreeBSD 10 and getting me to resolve conflicts between my existing configuration files and the new versions. Part way through, we are asked to reboot and then continue the upgrade process using the freebsd-update command utility. PC-BSD failed to reboot and, in fact, the boot loader no longer found any operating systems to run.” Ouch! I’m not sure on the particular commands used, but to lose the boot-loader indicates something went horribly wrong. There is good news in this though. After the pain experienced in the 9.X upgrade process, 11.0 has been vastly improved to help fix this going forward. The updater is also self-updating, which means future changes to tools such as package can be accounted for in previously released versions. Moving on to OpenBSD, Jesse had much better luck: > “The documentation provided explains how to upgrade OpenBSD 5.8 to version 5.9 step-by-step and the instructions worked exactly as laid out. Upgrading requires two reboots, one to initiate the upgrade process and one to boot into the new version of OpenBSD. Upgrading the base operating system took approximately ten minutes, including the two reboots. Upgrading the third-party packages took another minute or two. The only quirk I ran into was that I had to manually update my repository mirror information to gain access to the new packages available for OpenBSD 5.9. If this step is not done, then the pkg_add package manager will continue to pull in packages from the old repository we set up for OpenBSD 5.8. “ A good read, and they covered some Linux distros such as Mint and OpenMandriva as well, if you want to find out how they fared. *** A curated list of awesome DTrace books, articles, videos, tools and resources (http://awesome-dtrace.com/) The website awesome-dtrace.com compiles a list of resources, including books, articles, videos, tools, and other resources, to help you get the most out of DTrace The list of books includes 2 open source books that are available on the web, and of course Brendan Gregg’s official DTrace book There are also cheat sheets, one-liner collections, and a set of DTrace war stories A breakdown of different PID providers and the userspace statically defined tracepoints The videos from DTrace.conf 2008, 2012, and soon 2016 And links to the tools to start using DTrace with your favourite programming language, including Erlang, Node.JS, Perl, PHP, Python, or Ruby There are also DTrace setups for MySQL/MariaDB, and PostreSQL Joyent has even written a mod_usdt DTrace module for the Apache web server This seems like a really good resource, and with the efforts of the new OpenDTrace project, to modernize the dtracetoolkit and make it more useful across the different supported operating systems, there has never been a better time to start learning DTrace *** Installing OpenBSD using a serial console with no external monitor (http://unix.stackexchange.com/questions/292891/how-can-i-install-openbsd-using-the-serial-console-without-external-monitor-wi) Have you found yourself needing to install OpenBSD from USB, but with a twist, as in no external monitor? Well somebody has and asked the question on stackexchange. The answer provided is quite well explained, but in a nut-shell the process involves downloading the USB image and making some tweaks before copying it to the physical media. Specifically with a couple of well-placed echo’s into boot.conf, the serial-port can be enabled and ready for use: echo "stty com0 115200" > /mnt/etc/boot.conf echo "set tty com0" >> /mnt/etc/boot.conf + After that, simply boot the box and you are ready to access the serial console and drive the installation as normal! #bsdhacks GSoC 2016 Reports: Split debug symbols for pkgsrc builds (https://blog.netbsd.org/tnf/entry/gsoc_2016_reports_split_debug) The NetBSD blog provides a status report on one of the GSoC projects that is nearing its midterm evaluation The project to split debugging data into separate pkgsrc packages, so that users can install the debugging symbols if they need them to debug a failing application The report is very detailed, and includes “A quick introduction to ELF and how debug information are stored/stripped off” It walks through the process of writing a simple example application, compiling it, and dealing with the debug data It includes a number of very useful diagrams, and a summary of what changes needed to be make to the pkgsrc makefile infrastructure With this as a recipe, someone should be able to do something quite similar for FreeBSD’s ports tree *** iXsystems iXsystems’ TrueNAS Firmware Update Delivers Compelling Performance, Replication, and Graphing Improvements (https://www.ixsystems.com/blog/ixsystems-truenas-firmware-update-delivers-compelling-performance-replication-graphing-improvements/) *** Interview - Jim Brown - jpb@jimby.name (mailto:jpb@jimby.name) FreeBSD+BBB Sprinkler System News Roundup From the past : A Research Unix Reader (http://www.cs.dartmouth.edu/~doug/reader.pdf) A paper by by Douglas McIlroy “Selected pages from the nine research editions of the UNIX® Programmer’s Manual illustrate the development of the system” “Accompanying commentary recounts some of the needs, events, and individual contributions that shaped this evolution.” Interesting insight into the evolution of the origin UNIX operating system *** Evolution of C programming practices – Unix 1973–2015 (http://kristerw.blogspot.com/2016/06/evolution-of-c-programming-practices.html) From the author of the recent post we covered, “20 years of NetBSD code bloat”, comes a new post “I found a recent paper that also looks at how the BSD code base has evolved, but from a very different perspective compared to my code-size investigation.” The paper "The Evolution of C Programming Practices: A Study of the Unix Operating System 1973–2015" investigates coding style, and tests seven hypotheses by looking at metrics (line length, number of volatile in the source code, etc.) in 66 releases of Unix from 1973 to 2014. The hypotheses are: > + Programming practices reflect technology affordances (e.g. developers may be more liberal with screen space when using high resolution displays) > + Modularity increases with code size > + New language features are increasingly used to saturation point > + Programmers trust the compiler for register allocation > + Code formatting practices converge to a common standard > + Software complexity evolution follows self correction feedback mechanisms > + Code readability increases and the result is that they seem to be true, as interpreted through the metrics. > “The data points for the releases have somewhat random dates. One issue is that the paper use each release's mean file date (the average of the files' last modification time) instead of the release date (that is why the graphs stop at November 2010, even though FreeBSD 10 was released in 2014). The idea is that this better reflects the age of the code base, but this has the effect of compressing some of the data points (especially the clustering around 1993-1994), and it makes the spline fitting even more suspect.” > “One other problem is that the original data used by the researchers seems to have incorrect timestamps. For example, 4.3BSD Net/1 was released in 1989, but is listed as 1993-12-25 in the paper. The same is true for at least the Net/2 release too, which was released in 1991, but the paper list it as 1993-07-02.” *** [old release pictures] openbsd 2.1 - 5.9, straight from theo's bookshelf. (https://twitter.com/blakkheim/status/747540167112671232) Speaking of old releases, our Producer JT picked up this gem at Southeast Linuxfest this year (https://twitter.com/q5sys/status/748003859012984837) Noah Axon shares a scan of his NetBSD 1.4 disc (https://plus.google.com/+NoahAxon/posts/VsiQhUn3tHb) Jan van den broek shares a pic of his FreeBSD 2.2.5 set (https://plus.google.com/101232368324501316985/posts/4QsaJE2KxXh) *** FreeBSD: Just in Time (https://bsdmag.org/just_in_time/) Another BSDMag goodie this week, we have a small article written by Jonathan Garrido which details their experience switching to FreeBSD for a NTP server. The article is short, but a good read: > “A Few years ago we had a time problem. Suddenly our linux NTP server, for a reason that I still do not know, started to fail giving us a lot of issues within all the equipment and services within our network. After a quick and brief meeting with management, I found out that there was not sufficient budget left for a fancy and well-suited appliance. So, with no time (literally) and no money to spend, I decided to give it a try and utilized a homemade open source solution, and the operating system of choice was FreeBSD 10.0.” “Now, let’s pause for a second. You may be thinking, why in the world is this guy doing this, when he has never installed a BSD machine in his life? The answer is very simple; here, in the Dominican Republic, in the heart of the Caribbean, FreeBSD has a very good reputation when it comes to reliability and security. In fact, there is some collective thought within the sysadmin community that says something like: “If you want to deal only once with a service, install it over FreeBSD.”” Jonathan then goes through some of the steps taken to initial deploy NTP services, but with that out of the way, he has a great summary: > “Fascinated with the whole experience, we migrate one of our internal dns servers to a second FreeBSD machine and at the moment of this writing we are testing haproxy, an open source load-balancing proxy into a another server with the same OS. > After all this, no time issues have been reported in the past 2 years, so at least for my environment, FreeBSD came just in time.“ *** Beastie Bits MiniBSD laptop computer (https://hackaday.io/project/643-minibsd-laptop-computer) The state of LibreSSL in FreeBSD (https://attilagyorffy.com/2016/07/02/the-state-of-libressl-in-freebsd/) Justin Sherrill is looking for someone willing to run a Go builder with DragonflyBSD (https://www.dragonflydigest.com/2016/07/01/18372.html) Tiny Unix tools for Windows (https://tinyapps.org/blog/windows/201606040700_tiny_unix_tools_windows.html) OpenBSD's doas added to the FreeBSD Ports Tree (http://www.freshports.org/security/doas/) ubuntuBSD 16.04 to feature a combo of BusyBox and OpenRC, no systemd (http://linux.softpedia.com/blog/ubuntubsd-16-04-will-feature-a-combination-of-busybox-and-openrc-but-no-systemd-505463.shtml) Syncast Podcast 4 : Curl, libcurl and the future of the web, with Daniel Stenberg (http://podcast.sysca.st/podcast/4-curl-libcurl-future-web-daniel-stenberg/) Feedback/Questions Harri - Using beadm / zfssnap (http://pastebin.com/qKeCd63F) Jonathan - bhyve vs Proxmox (http://pastebin.com/EhXDwbWQ) Mohammad - Bhyve gfx passthrough (http://pastebin.com/ZCNk4Bga) Jeremy - Shapshots and more Snapshots (http://pastebin.com/xp7nzEYa) Ron - Microphone (http://pastebin.com/H2xr53CR) ***
149: The bhyve has been disturbed, and a wild Dexter appears!
Today on the show, we are going to be chatting with Michael Dexter about a variety of topics, but of course including bhyve! That plus This episode was brought to you by Headlines NetBSD Introduction (https://bsdmag.org/netbsd_intr/) We start off today’s episode with a great new NetBSD article! Siju Oommen George has written an article for BSDMag, which provides a great overview of NetBSD’s beginnings and what it is today. Of course you can’t start an article about NetBSD without mentioning where the name came from: “The four founders of the NetBSD project, Chris Demetriou, Theo de Raadt, Adam Glass, and Charles Hannum, felt that a more open development model would benefit the project: one centered on portable, clean and correct code. They aimed to produce a unified, multi-platform, production-quality, BSD-based operating system. The name “NetBSD” was suggested by de Raadt, based on the importance and growth of networks, such as the Internet at that time, the distributed and collaborative nature of its development.” From there NetBSD has expanded, and keeping in line with its motto “Of course it runs NetBSD” it has grown to over 57 hardware platforms, including “IA-32, Alpha, PowerPC,SPARC, Raspberry pi 2, SPARC64 and Zaurus” From there topics such as pkgsrc, SMP, embedded and of course virtualization are all covered, which gives the reader a good overview of what to expect in the modern NetBSD today. Lastly, in addition to mentioning some of the vendors using NetBSD in a variety of ways, including Point-Of-Sale systems, routers and thin-clients, you may not have known about the research teams which deploy NetBSD: NASA Lewis Research Center – Satellite Networks and Architectures Branch use NetBSD almost exclusively in their investigation of TCP for use in satellite networks. KAME project – A research group for implementing IPv6, IPsec and other recent TCP/IP related technologies into BSD UNIX kernels, under BSD license. NEC Europe Ltd. established the Network Laboratories in Heidelberg, Germany in 1997, as NEC’s third research facility in Europe. The Heidelberg labs focus on software-oriented research and development for the next generation Internet. SAMS-II Project – Space Acceleration Measurement System II. NASA will be measuring the microgravity environment on the International Space Station using a distributed system, consisting of NetBSD.“ My condolences, you’re now the maintainer of a popular open source project (https://runcommand.io/2016/06/26/my-condolences-youre-now-the-maintainer-of-a-popular-open-source-project/) A presentation from a Wordpress conference, about what it is like to be the maintainer of a popular open source project The presentation covers the basics: Open Source is more than just the license, it is about community and involvement The difference between Maintainers and Contributors It covers some of the reasons people do not open up their code, and other common problems people run into: “I'm embarrassed by my code” (Hint: so is everyone else, post it anyway, it is the best way to learn) “I'm discouraged that I can't finish releases on time” “I'm overwhelmed by the PR backlog” “I'm frustrated when issues turn into flamewars” “I'm overcommitted on my open source involvement” “I feel all alone” Each of those points is met with advice and possible solutions So, there you have it. Open up your code, or join an existing project and help maintain it *** FreeBSD Committer Allan Jude Discusses the Advantages of FreeBSD and His Role in Keeping Millions of Servers Running (http://www.hostingadvice.com/blog/freebsd-project-under-the-hood/) An interesting twist on our normal news-stories today, we have an article featuring our very own Allan Jude, talking about why FreeBSD and the advantages of working on an open-source project. “When Allan started his own company hosting websites for video streaming, FreeBSD was the only operating system he had previously used with other hosts. Based on his experience and comfort with it, he trusted the system with the future of his budding business.A decade later, the former-SysAdmin went to a conference focused on the open-source operating system, where he ran into some of the folks on its documentation team. “They inspired me,” he told our team in a recent chat. He began writing documentation but soon wanted to contribute improvements beyond the docs.Today, Allan sits as a FreeBSD Project Committer. It’s rare that you get to chat with someone involved with a massive-scale open-source project like this — rare and awesome.” From there Allan goes into some of the reasons “Why” FreeBSD, starting with Code Organization being well-maintained and documented: “The FreeBSD Project functions like an extremely well-organized world all its own. Allan explained the environment: “There’s a documentation page that explains how the file system’s laid out and everything has a place and it always goes in that place.”” + In addition, Allan gives us some insight into his work to bring Boot-Environments to the loader, and other reasons why FreeBSD “just makes sense” + In summary Allan wraps it up quite nicely: “An important take-away is that you don’t have to be a major developer with tons of experience to make a difference in the project,” Allan said — and the difference that devs like Allan are making is incredible. If you too want to submit the commit that contributes to the project relied on by millions of web servers, there are plenty of ways to get involved! We’re especially talking to SysAdmins here, as Allan noted that they are the main users of FreeBSD. “Having more SysAdmins involved in the actual build of the system means we can offer the tools they’re looking for — designed the way a SysAdmin would want them designed, not necessarily the way a developer would think makes the most sense” A guide to saving electricity and time with poudriere and bhyve (http://justinholcomb.me/blog/2016/07/03/poudriere-in-bhyve-and-bare-metal.html) “This article goes over running poudriere to built packages for a Raspberry Pi with the interesting twist of running it both as a bhyve guest and then switching to running on bare metal via Fiber Channel via ctld by sharing the same ZFS volume.” “Firstly, poudriere can build packages for different architectures such as ARM. This can save hours of build time compared to building ports from said ARM device.” “Secondly, let’s say a person has an always-on device (NAS) running FreeBSD. To save power, this device has a CPU with a low clock-rate and low core count. This low clock-rate and core count is great for saving power but terrible for processor intensive application such as poudriere. Let’s say a person also has another physical server with fast processors and a high CPU count but draws nearly twice the power and a fan noise to match.” “To get the best of both worlds, the goal is to build the packages on the fast physical server, power it down, and then start the same ZFS volume in a bhyve environment to serve packages from the always-on device.” The tutorial walks through setting up ‘ahost’, the always on machine, ‘fhost’ the fast but noisy build machine, and a raspberry pi It also includes creating a zvol, configuring iSCSI over fibre channel and exporting the zvol, booting an iSCSI volume in bhyve, plus installing and setting up poudriere This it configures booting over fibre channel, and cross-building armv6 (raspberry pi) packages on the fast build machine Then the fast machine is shut down, and the zvol is booted in bhyve on the NAS Everything you need to know to make a hybrid physical/virtual machine The same setup could also work to run the same bhyve VM from either ahost or fhost bhyve does not yet support live migration, but when it does, having common network storage like the zvol will be an important part of that *** Interview - Michael Dexter - editor@callfortesting.org (mailto:editor@callfortesting.org) / @michaeldexter (https://twitter.com/michaeldexter) The RoloDexter *** iXSystems Children's Minnesota Star Studio Chooses iXsystems' TrueNAS Storage (https://www.youtube.com/watch?v=FFbdQ_05e-0) *** News Roundup FreeBSD Foundation June 2016 Update (https://www.freebsdfoundation.org/wp-content/uploads/2016/06/FreeBSD-Foundation-June-2016-Update.pdf) The FreeBSD Foundation’s June newsletter is out Make sure you submit the FreeBSD Community Survey (https://www.surveymonkey.com/r/freebsd2016) by July 7th: In addition to the opening message from the executive director of the foundation, the update includes details to sponsored work on the FreeBSD VM system, reports from a number of conferences the Foundation attended, including BSDCan The results of the foundation's yearly board meeting People the foundation recognized for their contributions to FreeBSD at BSDCan And an introduction to their new “Getting Started with FreeBSD” project *** [How-To] Building the FreeBSD OS from scratch (http://www.all-nettools.com/forum/showthread.php?34422-Building-the-FreeBSD-OS-from-scratch) A tutorial over at the All-NetTools.com forums that walks through building FreeBSD from scratch I am not sure why anyone would want to build Xorg from source, but you can It covers everything in quite a bit of detail, from the installation process through adding Xorg and a window manager from source It also includes tweaking some device node permissions for easier operation as a non-root user, and configuring the firewall *** Window Systems Should Be Transparent (http://doc.cat-v.org/bell_labs/transparent_wsys/) + Rob Pike of AT&T Labs writes about why Window Systems should be transparent This is an old paper (undated, but I think from the late 80s), but may contain some timeless insights “UNIX window systems are unsatisfactory. Because they are cumbersome and complicated, they are unsuitable companions for an operating system that is appreciated for its technical elegance” “A good interface should clarify the view, not obscure it” “Mux is one window system that is popular and therefore worth studying as an example of good design. (It is not commercially important because it runs only on obsolete hardware.) This paper uses mux as a case study to illustrate some principles that can help keep a user interface simple, comfortable, and unobtrusive. When designing their products, the purveyors of commercial window systems should keep these principles in mind.” There are not many commercial window systems anymore, but “open source” was not really a big thing when this paper was written *** Roger Faulkner, of Solaris fame passed away (http://permalink.gmane.org/gmane.comp.standards.posix.austin.general/12877) “RIP Roger Faulkner: creator of the One and True /proc, slayer of the M-to-N threading model -- and the godfather of post-AT&T Unix” @bcantrill: Another great Roger Faulkner story (https://twitter.com/bcantrill/status/750442169807171584) The story of how pgrep -w saved a monitor -- if not a life (https://news.ycombinator.com/item?id=4306515) @bcantrill: With Roger Faulkner, Tim led an engineering coup inside Sun that saved Solaris circa 2.5 (https://twitter.com/bcantrill/status/750442169807171584) *** Beastie Bits: Developer Ed Maste is requesting information from those who are users of libvgl. (https://lists.freebsd.org/pipermail/freebsd-stable/2016-June/084843.html) HEADS UP: DragonFly 4.5 world reneeds rebuilding (http://lists.dragonflybsd.org/pipermail/users/2016-June/249748.html) Chris Buechler is leaving the pfSense project, the entire community thanks you for your many years of service (https://blog.pfsense.org/?p=2095) GhostBSD 10.3-BETA1 now available (http://ghostbsd.org/10.3_BETA1) DragonFlyBSD adds nvmectl (http://lists.dragonflybsd.org/pipermail/commits/2016-June/500671.html) OPNsense 16.1.18 released (https://opnsense.org/opnsense-16-1-18-released/) bhyve_graphics hit CURRENT (https://svnweb.freebsd.org/base?view=revision&revision=302332) BUG Update FreeBSD Central Twitter account looking for a new owner (https://twitter.com/freebsdcentral/status/750053703420350465) NYCBUG meeting : Meet the Smallest BSDs: RetroBSD and LiteBSD, Brian Callahan (http://lists.nycbug.org/pipermail/talk/2016-July/016732.html) NYCBUG install fest @ HOPE (http://lists.nycbug.org/pipermail/talk/2016-June/016694.html) SemiBUG is looking for presentations for September and beyond (http://lists.nycbug.org/pipermail/semibug/2016-June/000107.html) Caleb Cooper is giving a talk on Crytpo at KnoxBUG on July 26th (http://knoxbug.org/content/2016-07-26) Feedback/Questions Leif - ZFS xfer (http://pastebin.com/vvASr64P) Zach - Python3 (http://pastebin.com/SznQHq7n) Dave - Versioning (http://pastebin.com/qkpjKEr0) David - Encrypted Disk Images (http://pastebin.com/yr7BUmv2) Eli - TLF in all the wrong places (http://pastebin.com/xby81NvC) ***
148: The place to B...A Robot!
This week on the show, Allan and I are going to be showing you a very interesting interview we did talking about using FreeBSD to drive This episode was brought to you by Headlines FreeBSD Core Team Election (https://www.freebsd.org/administration.html#t-core) Core.9 has been elected, and will officially take over from Core.8 on Wednesday, 6 July 2016 Many thanks to the outgoing members of the core team for their service over the last 2 years 214 out of 325 eligible voters (65.8%) cast their votes in an election counting 14 candidates. The top nine candidates are, in descending order of votes received: 180 84.1% Ed Maste (incumbent) 176 82.2% George V. Neville-Neil (incumbent) 171 79.9% Baptiste Daroussin (incumbent) 168 78.5% John Baldwin 166 77.6% Hiroki Sato (incumbent) 147 68.7% Allan Jude 132 61.7% Kris Moore 121 56.5% Benedict Reuschling 108 50.5% Benno Rice There was no tie for ninth. BSDNow and the entire community would also like to extend their thanks to all those who stood for election to the core team Next week’s core meeting will encompass the members of Core.8 and Core.9, as responsibility for any outstanding items will be passed from outgoing members of core to the new incoming members *** Why I run OpenBSD (http://deftly.net/posts/2016-05-31-why-i-run-openbsd.html) This week we have a good article / blog post talking about why the posted has moved to OpenBSD from Linux. “One thing I learned during my travels between OSs: consistency is everything. Most operating systems seem to, at least, keep a consistent interface between themselves and binaries / applications. They do this by keeping consistent APIs (Application Programming Interfaces) and ABIs (Application Binary Interfaces). If you take a binary from a really old version of Linux and run or build it on a brand-spanking new install of Linux, it will likely Just Work™. This is great for applications and developers of applications. Vendors can build binaries for distribution and worry less about their product working when it gets out in the wild (sure this binary built in 2016 will run on RedHat AS2.1!!).“ The author then goes through another important part of the consistency argument, with what he calls “UPI” or “User Program Interfaces”. In other words, while the ABI may be stable, what about the end-user tooling that the user directly has to interact with on a daily basis? “This inconsistency seems to have come to be when Linux started getting wireless support. For some reason someone (vendors, maybe?) decided that ifconfig wasn’t a good place to let users interact with their wireless device. Maybe they felt their device was special? Maybe there were technical reasons? The bottom line is, someone decided to create a new utility to manage a wireless device… and then another one came along… pretty soon there was iwconfig(8), iw(8), ifconfig(8), some funky thing that let windows drivers interface with Linux.. and one called ip(8) I am sure there are others I am forgetting, but I prefer to forget. I have moved onto greener pastures and the knowledge of these programs no longer serves me.” The article then goes through the rundown of how he evaluated the various BSD’s and ultimately settled on OpenBSD: “OpenBSD won the showdown. It was the most complete, simple, and coherent system. The documentation was thorough, the code was easy to follow and understand. It had one command to configure all of the network interfaces! I didn’t have wireless, but I was able to find a cheap USB adapter that worked by simply running man -k wireless and reading about the USB entries. It didn’t have some of the applications I use regularly, so I started reading about ports (intuitively, via man ports!).” The ultimate NetBSD Router (http://blog.tbrodel.me/2016/#netbsd-router) “So yesterday I spent the day setting up a new firewall at home here, based off of this BSD Now tutorial. Having set up a couple of OpenBSD routers before, either based on old laptops, bulky old power-sucking desktops or completely over-specced machines like the Intel NUC, I wanted to get some kind of BSD onto a low-powered ARM board and use that instead.” “I've had a couple of Cubietrucks lying around for a while now, I've used them in a couple of art installations, running Debian and Pure Data, but over all they've been a bit disappointing. It's more the manufacturer's fault but they require blobs for the graphics and audio, which Debian won't allow, so as a multimedia board they're dud for video, and only passable for audio work with a usb sound card. So they've been collecting dust.” “Only thing missing is a second NIC, luckily I had an Apple USB->Ethernet dongle lying around, which when I bought it was the cheapest thing I could find on eBay that OpenBSD definitely supported. There, and on NetBSD, it's supported by the axe(4) driver. USB 2.0 works fine for me as I live in Australia and my ISP can only give me 30Mbps, so this should do for the forseeable future.” + The article then walks through installing and configuring NetBSD + Configuration includes: pf, unbound, and dhcpd “This project has been really fun, I started with basically no experience with NetBSD and have finished with a really useful, low-powered and robust appliance. It's a testament to the simplicity of the NetBSD system, and the BSD design principles in general, that such a novice as myself could figure this out. The NetBSD project has easily the most polished experience on Allwinner ARM boards, even Debian doesn't make it this easy. It's been a joy running the system, it has the bits I love from OpenBSD; ksh(1), tmux(1), an http daemon in base and of course, pf(4). This is mixed with some of the pragmatism I see in FreeBSD; a willingness to accept blobs if that really is the only way to boot, or get audio, or a video console.” bhyve-Bootable Boot Environments (http://callfortesting.org/bhyve-boot-environments/) We have a lengthy article also today from our friend Michael Dexter, who asks the basic question “What if multibooting and OS upgrades weren't horrible?” No doubt if you’ve been a frequent listener to this show, you’ve heard Allan or Myself talking about ZFS Boot Environments, and how they can “change your life”. Well today Michael goes further into detail on how the BE’s work, and how they can be leveraged to do neat things, like installing other versions of an operating system from the original running system. “If you are reading this, you have probably used a personal computer with a BSD or GNU/Linux operating system and at some point attempted to multiboot between multiple operating systems on the same computer. This goal is typically attempted with complex disk partitioning and a BSD or GNU/Linux boot loader like LILO or GRUB, plus several hours of frustrating experimentation and perhaps data loss. While exotic OS experimentation has driven my virtualization work since the late 1990s, there are very pragmatic reasons for multibooting the same OS on the same hardware, notable for updates and failback to "known good" versions. To its credit, FreeBSD has long had various strategies including the NanoBSD embedded system framework with primary and secondary root partitions, plus the nextboot(8) utility for selecting the "next" kernel with various boot parameters. Get everything set correctly and you can multiboot "with impunity". “That's a good start, and over time we have seen ZFS "boot environments" be used by PC-BSD and FreeNAS to allow for system updates that allow one to fall back to previous versions should something go wrong. Hats off to these efforts but they exist in essentially purpose-built appliance environments. I have long sensed that there is more fun to be had here and a wonderful thing happened with FreeBSD 10.3 and 11.0: Allan Jude added a boot environment menu to the FreeBSD loader” From here Michael takes us through the mechanical bits of actually creating a new ZFS dataset (BE) and performing a fresh FreeBSD 10.3 installation into this new boot-environment. The twist comes at the end, where he next sets up the BE to be a root NFS for booting in bhyve! This is interesting and gives you a way to test booting into your new environment via a VM, before rebooting the host directly into it. *** Interview - Edicarla Andrade & Vinícius Zavam - @egypcio (https://twitter.com/egypcio) BSD-Powered Robots News Roundup Tomohiro Kasumi explains what “@@” means, in the context of the Hammer filesystem (http://lists.dragonflybsd.org/pipermail/users/2016-June/249717.html) A post from the Dragonfly users’ mailing list about what the @@ construct means in the Hammer filesystem “@@ represents the existence of a PFS which is logically separated pseudo filesystem space within HAMMER's B-Tree” “HAMMER only has 1 large B-Tree per filesystem (not per PFS), so all the PFS exist within that single B-Tree. PFS are separated by localization parameter which is one of the B-Tree keys used to lookup the tree.” Each substring in "@@-1:00001" means: "@@" means it's a PFS or snapshot. "-1" means it's a master. ":" is just a separator. "00001" means it's PFS#1, where PFS#0 is the default PFS created on newfs. There is no "00000" because that's what's mounted on /HAMMER. PFS# is used for localization parameter. “Localization parameter has the highest priority when inserting or looking up B-Tree elements, so fs elements that belong to the same PFS# tend to be localized (clustered) within the B-Tree” There is also a note about how snapshots are named: "@@0x00..." A user points out that having : in the path can confuse some applications, such as in the case of adding the current directory or a relative path to the $PATH environment variable, which is a colon delimited list of paths This seems quite a bit more confusing that the datasets created by ZFS, but they might have other useful properties *** FreeBSD 11.0 nearing RC1 (https://www.freebsd.org/releases/11.0R/schedule.html) We’ve all been eagerly awaiting the pending release of FreeBSD 11.0, and the schedule has now been updated! The first release candidate is slated for July 29th! If all goes well (and we stick to schedule) there will be another RC2 and possible RC3 release, before 11.0 officially drops near the end of August. Start playing with those builds folks, be sure to send your feedback to the team to make this the best .0 release ever! *** TensorFlow on FreeBSD (http://ecc-comp.blogspot.com/2016/06/tensorflow-on-freebsd.html) Next we have a blog post about the experience of a “new” FreeBSD user trying to deploy some non-ported software to his new system. Specifically he was interested in running TensorFlow, but not doing a port himself, because in his words: “First, I apologize for not supplying a port archive myself. After reading the FreeBSD handbook for creating a port, it's too complex of a task for me right now. I've only been using FreeBSD for two weeks. I would also not like to waste anyone's time giving them a terrible port archive and mess up their system.” First of all, good ports are often born out of bad ports! Don’t let the porting framework daunt you, give it a go, since that's the only way you are going to learn how to write “good” ports over time. The porters-handbook is a good first place to start, plus the community usually is very helpful in providing feedback. He then walks us through the changes made to the TensorFlow code (starting with the assumption that OSX was a good “flavor” to begin porting from) and ultimately compiling. This ends up with the creation of a pip package which works! A good tutorial, and also very similar to what goes on in the porting process. With this write-up perhaps somebody will take up creating a port of it… hint hint! *** NetBSD: A New Beginning? (http://jamesdeagle.blogspot.ca/2016/06/netbsd-new-beginning.html) We don’t get enough NetBSD news at times, but this post by James Deagle talks about his adventure with NetBSD 7.0 and making it his “new beginning” “After a few months of traipsing around the worlds of SunOS and Linux, I'm back to NetBSD for what I hope will be a lengthy return engagement. And while I'm enamored of NetBSD for all the previously-mentioned reasons, I'm already thinking ahead to some problems to solve, some of which have also been mentioned before.” He then goes through and lists some of the small nits he’s still running into during the daily workflow YouTube audio - Specifically he mentions that no audio is playing, but wonders if Flash plays some part. (Ideally you’re not using Flash though, in which case you need to check the audio backend FF is using. Try PulseAudio since it seems the best supported. If pulse is already enabled, install ‘pavucontrol’ to make sure audio is playing to the correct sound device) Slow gaming performance (TuxKart and Celestia) - Check DRI / Xorg? Or is it CPU bound? Lastly some unspecified Wireless issues, which typically end up being driver related. (Or use another chipset) Beastie Bits Reproducible NetBSD? 77.7% of the way there (https://reproducible.debian.net/netbsd/netbsd.html) Create FreeBSD virtual machine using qemu. Run the VM using xhyve. (https://gist.github.com/zg/38a3afa112ddf7de4912aafc249ec82f) FreeBSD PowerPC 32bit pkg repository (unofficial). ~19,500 packages, more to come (https://joshcummings.net/pub/FreeBSD) NetBSD machines at Open Source Conference 2016 Gunma (http://mail-index.netbsd.org/netbsd-advocacy/2016/05/16/msg000706.html) Adam Leventhal (of ZFS and DTrace) does an analysis of APFS (http://arstechnica.com/apple/2016/06/a-zfs-developers-analysis-of-the-good-and-bad-in-apples-new-apfs-file-system/) SemiBug June meeting summary (http://lists.nycbug.org/pipermail/semibug/2016-June/000106.html) KnoxBug Meeting (http://knoxbug.org/content/2016-07-26) Feedback/Questions Andrew - iocage (http://pastebin.com/nuYTzaG6) Florian - Arm + GitHub (http://pastebin.com/PzY68hNS) Clint - Synth (http://pastebin.com/JESGZjLu) Leonardo - Translations (http://pastebin.com/b4LAiPs4) Zachary - Moving things to VMs (http://pastebin.com/VRc8fvBk) ***
147: Release all the things!
On this episode of BSDNow, we will be talking to Glen Barber and Peter Wemm of the FreeBSD RE and Cluster Admin teams! That plus our This episode was brought to you by Headlines 2016 FreeBSD Community Survey (https://www.surveymonkey.com/r/freebsd2016) We often get comments from our listeners, “I’m not a developer, how can I help out”? Well today is your chance to do something. The FreeBSD Foundation has its 2016 Community Survey online, where they are asking for feedback from you! I just did the survey, it’ll take you about 5 minutes, but gives you a chance to provide valuable feedback to the foundation about things that are important to you. Be sure to answer in as much detail as possible and the foundation will review and use this feedback for its operations going forward. *** ART, OpenBSDs new routing table, single thread performances (http://www.grenadille.net/post/2016/06/17/ART-single-thread-performances) OpenBSD has changed the way routes are looked up in the kernel as part of their path to an SMP networking stack The “Allotment Routing Table” (ART) is a performance tradeoff, where more memory is used to store the routing table, in exchange for faster lookups With this new arrangement, a full BGP routing table will grow from 130MB to 180MB of memory “ART is a free multibit trie based routing table. To keep it simple, it can be seen as using more memory for fewer CPU cycles. In other words, we get a faster lookup by wasting memory. The original paper (http://www.hariguchi.org/art/art.pdf) presents some performance comparisons between two ART configurations and the BSD Radix. But how does this apply to OpenBSD?” “I asked Hrvoje Popovski to run his packet forwarding test on his Xeon box (E5-2620 v2 @ 2.10GHz, 2400.34 MHz) with ix(4) (82599) interfaces. The test setup consist of three machines with the OpenBSD box in the middle” “The simulations have been performed with an OpenBSD -current from June 9th. The machine is configured with pf(4) disabled in order to force a single route lookup for every IPv4 packet. Based on the result of the lookup the kernel decide if it should forward, deliver or drop the packet” *** BSDCan 2016 Playlist (https://www.youtube.com/playlist?list=PLeF8ZihVdpFfoEV67dBSrKfA8ifpUr6qC) The complete set of videos from BSDCan is online and ready to be consumed Remember the good-ole days where we would wait months (or years) to get videos posted from conferences? Well, who are we kidding, some conferences STILL do that, but we can’t count BSDCan among them. Only two weeks out from this years exciting BSDCan, and all the videos have now landed on YouTube. Granted, this is no substitute for actually being at the conference, but even if you attended you probably missed quite a few of the talks. There are no videos of the hallway track, which is the best part of the conference Except the dinner discussion of course. and don’t forget the hacker lounge *** Should you be scared of Unix signals? (http://jvns.ca/blog/2016/06/13/should-you-be-scared-of-signals/) Do you know much about UNIX Signals? Are you afraid of their complexity? Do you know there are signals other than SIGKILL? This article talks about the practical implications of signals from a programming perspective The things you need to consider when dealing with signals Basically, you register a “signal handler”, the function that will be run when a signal arrives As you program is running, if a signal arrives, your program will be interrupted. Its current state will be saved and any system calls in progress will return EINTR (Error, Interrupted), then your signal handler will be run. Once the signal handler is complete, the state of your application will be restored, and execution will resume As long as your program properly handles this interruption, and errors that might result from it (getting EINTR from a read() call, instead of the data you expected), then everything should be fine. Of course, you need to be careful what you do inside your signal handler, as if you modify any variables or state in your application, it might be very confused when it resumes. *** Interview - Glen and Peter- News Roundup Unik - The Unikernel Compilation and Deployment Platform (uses NetBSD's Rump) (https://github.com/emc-advanced-dev/unik) We’ve talked a bit about NetBSD’s RUMP (unikernel) in the past, including articles on how to deploy services using it. Now we have an interesting project which makes the process super-easy, and dare-we-say almost “Docker-Like?” The Unik project has a fairly complete walkthrough right on their GitHub project page, including details on installation and creating your own unikernel containers. In addition, it provides instructions on boot-strapping your own Go/Node.js/Python/Java applications, and supports out of Box VCenter / AWS / Qemu / VirtualBox providers. *** PkgSrc 50th Release Highlights () pkgsrc is celebrating its 50th release, and to highlight this, they have posted a series of interviews from people who have been active in the project pkgsrc 50th release interviews - Jonathan Perkin (http://blog.netbsd.org/tnf/entry/pkgsrc_50th_release_interviews_jonathan) pkgsrc 50th release interviews - Ryo ONODERA (http://blog.netbsd.org/tnf/entry/pkgsrc_50th_release_interviews_ryo) pkgsrc 50th release interviews - Joerg Sonnenberg (http://blog.netbsd.org/tnf/entry/pkgsrc_50th_release_interview_with) pkgsrc 50th release interviews - Sevan Janiyan (https://blog.netbsd.org/tnf/entry/pkgsrc_50th_release_interviews_sevan) *** Migrating to FreeBSD from Solaris 11 (http://justinholcomb.me/blog/2016/02/28/migration-to-freebsd-part1.html) Part 2 (http://justinholcomb.me/blog/2016/03/12/migration-to-freebsd-part2.html) Part 3 (http://justinholcomb.me/blog/2016/03/19/migration-to-freebsd-part3.html) Part 4 (http://justinholcomb.me/blog/2016/03/26/migration-to-freebsd-part4.html) Part 5 (http://justinholcomb.me/blog/2016/04/03/migration-to-freebsd-part5.html) *** How to chroot www/firefox on NetBSD (https://github.com/alnsn/localpkgsrc/tree/master/firefox-chroot) Looking for a jail-like method of running FireFox on NetBSD? (Or possibly other BSDs?) We have a github repo with details on how to setup and run FireFox using a chroot using a “webuser” account for safety. Think of this as a jail alternative, may be useful on systems with no jail support. Of interest is the method used to do X forwarding. It uses Xorg TCP listen option (which is often off by default for security reasons). Perhaps SSH X forwarding would be a better alternative. (Or nullfs mounts of /tmp) *** Beastie Bits Tredly - V1 Release Candidate (https://github.com/tredly/tredly/releases/tag/v1.0.0-rc.1) Call for Testing - ypldap testing against OpenLDAP and Microsoft Active Directory (http://lists.freebsd.org/pipermail/freebsd-current/2016-June/061775.html) BSD Magazine, June 2016 Out Now (https://bsdmag.org/) Hammer2 - Add xxhash to H2 and throw in debug stuff for performance testing (http://lists.dragonflybsd.org/pipermail/commits/2016-June/500610.html) chyves pre-announcement (http://justinholcomb.me/blog/2016/06/14/chyves-project-preannouncement.html) *** Feedback/Questions Michael - Versioning (http://pastebin.com/1hpGrmuL) Michael - Removing Encryption (http://pastebin.com/2PkrMGGx) Bostjan - PC-BSD Questions (http://pastebin.com/q5VdmNxG) Fong - ZFS Rollback (http://pastebin.com/2aedLV7d) Jochen - Docker on FBSD (http://pastebin.com/dneVZkXc) ***
146: Music to Beastie’s ears
Kris is on vacation this week, so allan flies solo, provides a recap of BSDCan & cover's a boatload of news including Microsoft This episode was brought to you by Headlines BSDCan Recap and Live Stream Videos (http://www.bsdcan.org/2016/) OpenBSD BSDCan 2016 papers now available (http://www.openbsd.org/papers) Allan’s slides (http://allanjude.com/bsd/BSDCan2016_-_GELIBoot.pdf) and Paper (http://allanjude.com/bsd/AsiaBSDCon2016_geliboot_pdf1a.pdf) Michael W Lucas presents Allan with a gift (https://www.youtube.com/watch?v=LFgxAHkrSTg) “FreeBSD Mastery: Advanced ZedFS” (http://blather.michaelwlucas.com/archives/2698) Highlighted Tweets: Groff Arrives at BSDCan (https://twitter.com/Keltounet/status/740344735194320896) FreeBSD Foundation recognizes the contributions of Bryan Drewery, Rod Grimes, Warren Block, & Gleb Smirnoff (https://twitter.com/freebsdfndation/status/742456950676393984) A moment of silence and shots in memory in Benjamin Perrault @creepingfur (https://twitter.com/__briancallahan/status/741854476340858880) @gvnn3 sells the FreeBSD Foundation shirt off of his back for Charity (https://twitter.com/Keltounet/status/741763867471155201) Michael W. Lucas asks Matt Ahrens how to pronounce ZFS, “You can pronounce ZFS however you like, but if you pronounce it 'reiserfs', people might be confused.” (https://twitter.com/cperciva/status/741375414967410688) Sysadmin T-Shirt (https://twitter.com/BSDCan/status/741420633007874050) FreeBSD Dev Summit ran out of room on the chalkboards listing accomplishments of 11.0 (https://twitter.com/SeanChittenden/status/740904105388978176) List of things people have or want for FreeBSD 12 (https://twitter.com/Keltounet/status/740928627471159296) Matt Ahrens signing Allan’s ZFS book (https://twitter.com/kprovst/status/741322268480049152?cn=bWVudGlvbg%3D%3D&refsrc=email) FreeBSD’s new marketing strategy (https://twitter.com/cperciva/status/741707948469157889) Charity Auction: systemd whoopie cushion (https://twitter.com/HippyWizard/status/741768670704066560) Embarass OpenBSD’s @HenningBrauer by donating $10 to charity for a selfie with him wearing a Linux t-shirt (https://twitter.com/juliefriday/status/741948048788586496) @GroffTheBSDGoat changes handlers, from @HenningBrauer to @GavinAtkinson (https://twitter.com/GroffTheBSDGoat/status/742415390798716928) Day 1 Video (https://www.youtube.com/watch?v=AOidjSS7Hsg) Day 2 Video (https://www.youtube.com/watch?v=z7pDnBO5wSM) Allan’s GELIBoot talk (day 2) (https://www.youtube.com/watch?v=z7pDnBO5wSM&feature=youtu.be&list=PLeF8ZihVdpFfoEV67dBSrKfA8ifpUr6qC&t=4440) *** Media Coverage of Microsoft + FreeBSD story (https://azure.microsoft.com/en-us/blog/freebsd-now-available-in-azure-marketplace/) Microsoft has released their own custom image of FreeBSD 10.3 for the Azure Cloud “This means that not only can you quickly bring-up a FreeBSD VM in Azure, but also that in the event you need technical support, Microsoft support engineers can assist.” “Microsoft is the publisher of the FreeBSD image in the marketplace rather than the FreeBSD Foundation. The FreeBSD Foundation is supported by donations from the FreeBSD community, including companies that build their solutions on FreeBSD. They are not a solution provider or an ISV with a support organization but rather rely on a very active community that support one another. In order to ensure our customers have an enterprise SLA for their FreeBSD VMs running in Azure, we took on the work of building, testing, releasing and maintaining the image in order to remove that burden from the Foundation. We will continue to partner closely with the Foundation as we make further investments in FreeBSD on Hyper-V and in Azure.” "It's quite a significant milestone for FreeBSD community and for Microsoft to publish a supported FreeBSD image on Azure Marketplace. We really appreciate Microsoft's commitment and investment in FreeBSD project". - Justin T. Gibbs, President of FreeBSD Foundation Microsoft took a FreeBSD 10.3-RELEASE image and added additional patches, most of which they have upstreamed but that were too late for the regular 10.3 release cycle. Rather than requiring users to use a snapshot of the stable/10 branch, which would complicate the user experience, and complicate the job of the Microsoft support engineers, they created their own “certified” release This allows Microsoft to selectively deploy errata fixes to the image as well It is not clear how this affects update mechanisms like freebsd-update(8) The Register (http://www.theregister.co.uk/2016/06/09/microsoft_freebsd/) The Inquirer (http://www.theinquirer.net/inquirer/news/2461070/microsoft-creates-own-distribution-of-freebsd-for-azure-developers) Infoworld (http://www.infoworld.com/article/3082090/open-source-tools/is-microsoft-publishing-its-own-freebsd-yes-and-no.html) The Hacker News (http://thehackernews.com/2016/06/microsoft-azure-freebsd.html) Windows Report (http://windowsreport.com/microsoft-freebsd-10-3-ready-made-vm-image-azure/) Windows Club (http://news.thewindowsclub.com/microsoft-freebsd-operating-system-84375/) *** Select works poorly (http://www.tedunangst.com/flak/post/select-works-poorly) “At the bottom of the OpenBSD man page for select is a little note. “Internally to the kernel, select() and pselect() work poorly if multiple processes wait on the same file descriptor.” There’s a similar warning in the poll man page. Where does this warning come from and what does it mean?” Ted found that at first glance, OpenBSD’s select() appears to be quite bad: “whenever some data gets written, we call wakeup(&selwait);. Based on what we’ve seen so far, one can conclude that this is likely to be inefficient. Every time any socket has some data available, we wake up every selecting process in the system. Works poorly indeed.” After further investigation, it turns out to not be quite as bad When the select() is first setup, the PID of the process that cares about the FD is recorded in the selinfo struct If a second process runs select() on the same FD, the SI_COLL (Select Collision) flag is set on the selinfo struct When selwakeup() is called, if SI_COLL is set, all select()ing processes are woken up, and the sysctl kern.nselcoll is incremented. If the flag is not set, and only a single PID is waiting for activity on that FD, only that process is woken up “This is not an intractable problem. kevent avoids it entirely. Other implementations may too. But practically, does it need to be solved? My laptop says it’s happened 43 times. A server with substantially more uptime says 0. Doesn’t seem so bad.” *** Interview - Hans Petter Selasky - hps@freebsd.org (mailto:hps@freebsd.org) / @twitter (https://twitter.com/user) Designing FreeBSD’s USB drivers, hooking up a piano to FreeBSD & more! *** News Roundup Timeline of libexpat random vulnerability (http://www.tedunangst.com/flak/post/timeline-of-libexpat-random-vulnerability) Do you use FreeBSD as web server? Why or why not? (https://news.ycombinator.com/item?id=11804565) 20 years of NetBSD code Bloat (http://kristerw.blogspot.sg/2016/05/20-years-of-netbsd-code-bloat.html) HP Chromebook 13 now booting OpenBSD (https://jcs.org/statuses/2016/06/08/740606952149942272/) UNIX for Poets (https://web.stanford.edu/class/cs124/lec/124-UnixForPoets.pdf) Comparing live version upgrade methods (https://distrowatch.com/weekly.php?issue=20160530#upgrades) My life with FreeBSD on a Thinkpad X220 (https://www.reddit.com/r/BSD/comments/4n3flx/my_life_with_freebsd_on_a_thinkpad_x220/)
145: At the Core of it all
It’s BSDCan time! Allan and I are both enjoying what is sure to be a super-busy week, but don’t think we’ve forgotten about This episode was brought to you by Interview - Benno Rice - benno@freebsd.org (mailto:benno@freebsd.org) / @jeamland (https://twitter.com/jeamland) Manager, OS & Networking at EMC Isilon Emily Dunham: Community Automation (https://www.youtube.com/watch?v=dIageYT0Vgg) iXsystems 1U Rackmount Server - 4 Bay Hot-Swap SAS/SATA Drive Bays 400W Redundant Power Supply - Single Socket Embedded CPU (48 cores) - 8 DIMM Slots with 16GB DIMMs for a total of 128GB RAM – Dual Gigabit LAN, Dual 10GbE SFP+ and 1 x 40Gb QSFP+ port, (1) PCI-E Expansion Slots + IPMI Dedicated LAN - Cavium ThunderX ARM CN8890 48 Core ThunderX CPU - 2.5GHz per core System has 128GB RAM, 4 x 2TB SATA HDD, Additional Intel i350 (2 x 1GbE) Beastie Bits file considered harmful (http://www.tedunangst.com/flak/post/file-considered-harmful) An open source talk on ZFS. “Intro to ZFS” as a set of open source slides for the community to build on, and to reuse. Go give this talk at your local conference. (https://github.com/problame/talkintrozfs2016) ARMv7 now has a bootloader (http://undeadly.org/cgi?action=article&sid=20160529145411) SHA256/512 speed improvements in FreeBSD 11 (https://svnweb.freebsd.org/base?view=revision&revision=300966) pkgsrc 50th release interviews - Joerg Sonnenberg (http://blog.netbsd.org/tnf/entry/pkgsrc_50th_release_interview_with) DFly versus PC-BSD on a Laptop (http://lists.dragonflybsd.org/pipermail/users/2016-May/249636.html) FreeBSD ifconfig can print subnet masks in CIDR or dotted-quad, finally (https://svnweb.freebsd.org/base?view=revision&revision=301059) Feedback/Questions Eli - Getting rid of ports? (http://pastebin.com/4Y6VYSyN) Morgan - Best way to admin jails? (http://pastebin.com/w8hsMtbc) Simon - Use existing pkgs in poudriere (http://pastebin.com/mqSJk0pP) Pete - Lots of Q’s (http://pastebin.com/1M7HLAXs) Van - Made the switch (http://pastebin.com/NTVBvtC5) ***
144: The PF life
It’s only one-week away from BSDCan, both Allan and I are excited to meet some of you in person! However, the show keeps on This episode was brought to you by Headlines dotSecurity 2016 - Theo de Raadt - Privilege Separation and Pledge (http://www.dotsecurity.io/) Video (https://www.youtube.com/watch?v=a_EYdzGyNWs) Slides (https://www.openbsd.org/papers/dot2016.pdf) Interested in Privilege Separation and security in general? If so, then you are in for a treat, we have both the video and slides from Theo de Raadt at dotSecurity 2016. Specifically the the talk starts off looking at Pledge (no copyright issues with the pictures I hope??) and how their NTP daemon uses it. After going through some internals, Theo reveals that around 10% of programs “pledged” so far were found to be trying to do actions outside of their security scope. On the future-work side, they mention going back and looking at OpenSSH privilege separation next, as well as working with other OS’s that may want pledge support. *** bhyve now supports UEFI GOP (https://lists.freebsd.org/pipermail/freebsd-virtualization/2016-May/004471.html) The log awaited UEFI GOP (Graphics Output Protocol (https://en.wikipedia.org/wiki/Unified_Extensible_Firmware_Interface#GOP)) features has landed in bhyve This provides emulated graphics via an internal VNC server, allowing users to have full graphical access to the guest OS This allows installation of Windows guests without needing to create a modified ISO with an unattended installation script The code has not actually landed in FreeBSD head yet, but has been committed to a project branch Following a few simple commands, you can compile the new bhyve binary on your -CURRENT system and get started right away This feature is expected to be included in the upcoming FreeBSD 11.0 This commit drop also brings with it: XHCI -- an emulated usb tablet device that provides exact mouse positioning in supported OSs PS2 mouse for fallback if the guest does not support XHCI (Windows 7) PS2 keyboard “The code has been tested with Windows 7/8/8.1/10 and Server 2k12/2k16, Ubuntu 15.10, and FreeBSD 10.3/11-CURRENT” “For VNC clients, TightVNC, TigherVNC, and RealVNC (aka VNC Viewer) have been tested on various hosts. The OSX VNC client is known not to work.” The VNC server supports an optional ‘wait’ parameter, that causes the VM to not actually boot until the VNC client connects, allowing you to interrupt the boot process if need be Related user blog post (http://justinholcomb.me/blog/2016/05/28/bhyve-uefi-gop-support.html) SVN commit (https://svnweb.freebsd.org/base?view=revision&revision=300829) *** zfsd lands in FreeBSD HEAD, in time for 11.0-RELEASE (https://svnweb.freebsd.org/base?view=revision&revision=300906) zfsd has been committed to FreeBSD -CURRENT in time to be included in FreeBSD 11.0 zfsd is the missing piece required to make ‘hot spares’ work properly in FreeBSD ZFS “zfsd attempts to resolve ZFS faults that the kernel can't resolve by itself. It listens to devctl(4) events, which is how the kernel notifies of events such as I/O errors and disk removals. Zfsd attempts to resolve these faults by activating or deactivating hotspares and onlining offline vdevs.” “The administrator never interacts with zfsd directly. Instead, he controls its behavior indirectly through zpool configuration. There are two ways to influence zfsd: assigning hotspares and setting pool properties. Currently, only the autoreplace property has any effect. See zpool(8) for details.” So, what example does it do? Device Removal: “When a leaf vdev disappears, zfsd will activate any available hotspare.” Device Arrival: “When a new GEOM device appears, zfsd will attempt to read its ZFS label, if any. If it matches a previously removed vdev on an active pool, zfsd will online it. Once resilvering completes, any active hotspare will detach automatically.” So if you disconnect a drive, then reconnect it, it will automatically be brought back online. Since ZFS is smart, the resilver will only have to copy data that has changed since the device went offline. “If the new device has no ZFS label but its physical path matches the physical path of a previously removed vdev on an active pool, and that pool has the autoreplace property set, then zfsd will replace the missing vdev with the newly arrived device. Once resilvering completes, any active hotspare will detach automatically.” If the new drive is in the same slot in your hot swap array as a failed device, it will be used as a replacement immediately. vdev degrade or fault events: “If a vdev becomes degraded or faulted, zfsd will activate any available hotspare. If a leaf vdev generates more than 50 I/O errors in a 60 second period, then zfsd will mark that vdev as FAULTED. zfs(4) will no longer issue any I/Os to it. zfsd will activate a hotspare if one is available.” Same for checksum errors. So if zfsd detects a drive is going bad, it brings the hotspare online before it is too late Spare addition: “If the system administrator adds a hotspare to a pool that is already degraded, zfsd will activate the spare.” Resilver complete: “zfsd will detach any hotspare once a permanent replacement finishes resilvering.” Physical path change: “If the physical path of an existing disk changes, zfsd will attempt to replace any missing disk with the same physical path, if its pool's autoreplace property is set.” In general, this tool means less reliance on the system administrator to keep the pool healthy *** W^X now mandatory in OpenBSD (http://undeadly.org/cgi?action=article&sid=20160527203200) We’ve talked a bit about W^X in the past. (Refresher: Memory being writable and executable at once) Well, this major security no-no is no-more on OpenBSD. Theo has committed a change which now prevents violations of this policy: “W^X violations are no longer permitted by default. A kernel log message is generated, and mprotect/mmap return ENOTSUP. If the sysctl(8) flag kern.wxabort is set then a SIGABRT occurs instead, for gdb use or coredump creation.” There are a few cases where you may still need W^X, which Theo points out can be enabled on a file-system basis. “W^X violating programs can be permitted on a ffs/nfs filesystem-basis, using the "wxallowed" mount option. One day far in the future upstream software developers will understand that W^X violations are a tremendously risky practice and that style of programming will be banished outright. Until then, we recommend most users need to use the wxallowed option on their /usr/local filesystem. At least your other filesystems don't permit such programs.” This is a great ability to grow, since now users can begin doing auditing of programs that violate this principle and making noise to upstream. *** Interview - Kristof Provost - kp@freebsd.org (mailto:kp@freebsd.org) @kprovst (https://twitter.com/kprovst) pf improvements on FreeBSD *** News Roundup GELI Support for the EFI Loader (https://ericmccorkleblog.wordpress.com/2016/05/28/freebsd-geli-support/) We’ve had Allan’s work to bring GELI support to the GPT / BIOS / ZFS loader for a while now, but the missing piece has been support for EFI. No longer, Eric McCorkle has posted a blog entry (with relevant github links) introducing us to his work to bring GELI encryption support to EFI. First the bad-news. This won’t make it into 11.0. (Maybe PC-BSD, TBD) Next he explains why this is more than just a new feature, but a re-factor of the EFI boot code: I have already written extensively about my EFI refactoring here. The reason for undertaking this effort, however, was driven by GELI support. Early in my work on this, I had implemented a non-EFI “providers” framework in boot1 in order to support the notion of disk partitions that may contain sub-partitions. This was deeply unsatisfying to me for several reasons: It implemented a lot of the same functionality that exists in the EFI framework. It involved implementing a GPT partition driver to deal with partition tables inside GELI partitions (GPT detection and support is guaranteed by the EFI spec). The interface between the EFI framework and the custom “providers” framework was awkward. The driver was completely boot1-specific, and exporting it to something like GRUB probably involved a total rewrite. Implementing it within loader was going to involve a lot of code duplication. There was no obvious was to pass keys between boot1, loader, and the kernel. With the issues known, Eric seems pleased with the results of the conversion so far: The GELI driver can be extracted from the FreeBSD codebase without too much trouble. While I was unable to go all the way to the EFI driver model, the only blocker is the bcache code, and once that is resolved, we can have hotplug support in the boot loader! The boot1 and loader codebases are now sharing all the backend drivers, and boot1 has been reduced to one very small source file. An interesting read, looking forward to playing with EFI more in the future! *** Faces of FreeBSD 2016: Michael W. Lucas (https://www.freebsdfoundation.org/blog/faces-of-freebsd-2016-michael-lucas/) On this edition of “Faces of FreeBSD”, Michael W Lucas tells the story of how he got started with FreeBSD After an amusing re-telling of his childhood (The words “Purina Monkey Chow” were mentioned), he then tells us how he got into BSD. His being thrown into the project may sound familiar to many: I came in at 11 PM one night and was told “The DNS administrator just got walked out the door. You’re the new lead DNS administrator. Make those servers work. Good luck.” From there (because he wanted more sleep), he began ripping out the systems that had been failing and waking him up at night. Good-bye UnixWare, Good-bye Solaris, hello BSD! A very amusing read, check it out! *** High Availability with PostgreSQL on FreeBSD (https://www.youtube.com/watch?v=ugct9-Mm7Ls) A talk by Sean Chittenden, who we interviewed previously on episode Episode 95 (http://www.bsdnow.tv/episodes/2015_06_24-bitrot_group_therapy) Explains how to setup Multi Data Center High Availability for PostgreSQL using consul Goes into how consul works, how it does the election, the gossip protocol, etc The HA setup uses DNS Failover, and the pros and cons of that approach are discussed Then he walks through the implementation details, and example configuration *** New FreeBSD i915 testing images (http://www.bsddesktop.com/images/) Still need users to test the Linux Kernel 4.6 DRM update to FreeBSD’s graphics stack Download the test image and write it to a USB stick and boot from it It will not modify your installed system, it runs entirely off of the USB drive Allows you to test the updated drivers without having to install the development branch on your device you can tell them that ATI/AMD support will be coming shortly and that stability has been steadily improving and that I'll do another announcement as soon as I've had a chance to test the newest Xorg bits *** Beastie Bits Comfortable on the CLI: Series Part 1 (https://www.cotcli.com/post/The-Very-Basics/) FreeBSD Booting on the Netgate uFW, a smaller-than-a-raspberry-pi dual port firewall (https://gist.github.com/gonzopancho/8e7df7a826e9a2949b36ed2a9d30312e) Picture of uFW (https://twitter.com/gonzopancho/status/737874921435594753) uFW OpenSSL Benchmarks (https://gist.github.com/gonzopancho/8f20b50487a4f7de56e99448866a147d) ***
143: One small step for DRM, one giant leap for BSD
This week on BSDNow, we have an interview with Matthew Macy, who has some exciting news to share with us regarding the state of graphics This episode was brought to you by Headlines How the number of states affects pf’s performance of FreeBSD (http://blog.cochard.me/2016/05/playing-with-freebsd-packet-filter.html) Our friend Olivier of FreeNAS and BSDRP fame has an interesting blog post this week detailing his unique issue with finding a firewall that can handle upwards of 4 million state table entries. He begins in the article with benchmarking the defaults, since without that we don’t have a framework to compare the later results. All done on his Netgate RCC-VE 4860 (4 cores ATOM C2558, 8GB RAM) under FreeBSD 10.3. “We notice a little performance impact when we reach the default 10K state table limit: From 413Kpps with 128 states in-used, it lower to 372Kpps.” With the initial benchmarks done and graphed, he then starts the tuning process by adjusting the “net.pf.states_hashsize”sysctl, and then playing with the number of states for the firewall to keep. “For the next bench, the number of flow will be fixed for generating 9800 pf state entries, but I will try different value of pf.states_hashsize until the maximum allowed on my 8GB RAM server (still with the default max states of 10k):” Then he cranks it up to 4 million states “There is only 12% performance penalty between pf 128 pf states and 4 million pf states.” “With 10M state, pf performance lower to 362Kpps: Still only 12% lower performance than with only 128 states” He then looks at what this does of pfsync, the protocol to sync the state table between two redundant pf firewalls Conclusions: There need to be a linear relationship between the pf hard-limit of states and the pf.stateshashsize; RAM needed for pf.stateshashsize = pf.stateshashsize * 80 Byte and pf.stateshashsize should be a power of 2 (from the manual page); Even small hardware can manage large number of sessions (it's a matter of RAM), but under too lot's of pressure pfsync will suffer. Introducing the BCHS Stack = BSD, C, httpd, SQLite (http://www.learnbchs.org/) Pronounced Beaches “It's a hipster-free, open source software stack for web applications” “Don't just write C. Write portable and secure C.” “Get to know your security tools. OpenBSD has systrace(4) and pledge(2). FreeBSD has capsicum(4).” “Statically scan your binary with LLVM” and “Run your application under valgrind” “Don't forget: BSD is a community of professionals. Go to conferences (EuroBSDCon, AsiaBSDCon, BSDCan, etc.)” This seems like a really interesting project, we’ll have to get Kristaps Dzonsons back on the show to talk about it *** Installing OpenBSD's httpd server, MariaDB, PHP 5.6 on OpenBSD 5.9 (https://www.rootbsd.net/kb/339/Installing-OpenBSDandsharp039s-httpd-server-MariaDB-PHP-56-on-OpenBSD-59.html) Looking to deploy your next web-stack on OpenBSD 5.9? If so this next article from rootbsd.net is for you. Specifically it will walk you through the process of getting OpenBSD’s own httpd server up and running, followed by MariaDB and PHP 5.6. Most of the setup is pretty straight-forward, the httpd syntax may be different to you, if this is your first time trying it out. Once the various packages are installed / configured, the rest of the tutorial will be easy, walking you through the standard hello world PHP script, and enabling the services to run at reboot. A good article for those wanting to start hosting PHP/DB content (wordpress anyone?) on your OpenBSD system. *** The infrastructure behind Varnish (https://www.varnish-cache.org/news/20160425_website.html) Dogfooding. It’s a term you hear often in the software community, which essentially means to “Run your own stuff”. Today we have an article by PKH over at varnish-cache, talking about what that means to them. Specifically, they recently went through a website upgrade, which will enable them to run more of their own stuff. He has a great quote on what OS they use:“So, dogfood: Obviously FreeBSD. Apart from the obvious reason that I wrote a lot of FreeBSD and can get world-class support by bugging my buddies about it, there are two equally serious reasons for the Varnish Project to run on FreeBSD: Dogfood and jails.Varnish Cache is not “software for Linux”, it is software for any competent UNIX-like operating system, and FreeBSD is our primary “keep us honest about this” platform.“ He then goes through the process of explaining how they would setup a new Varnish-cache website, or upgrade it. All together a great read, and if you are one of the admin-types, you really should pay attention to how they build from the ground up. Some valuable knowledge here which every admin should try to replicate. I can not reiterate the value of having your config files in a private source control repo strongly enough The biggest take-away is: “And by doing it this way, I know it will work next time also.” *** Interview - Matt Macy - mmacy@nextbsd.org (mailto:mmacy@nextbsd.org)Graphics Stack Update (https://lists.freebsd.org/pipermail/freebsd-x11/2016-May/017560.html) News Roundup Followup on packaging base with pkg(8) (https://lists.freebsd.org/pipermail/freebsd-pkgbase/2016-May/000238.html) In spite of the heroic last minute effort by a team of contributors, pkg’d base will not be ready in time for FreeBSD 11.0 There are just too many issues that were discovered during testing The plan is to continue using freebsd-update in the meantime, and introduce a pkg based upgrade mechanism in FreeBSD 11.1 With the new support model for the FreeBSD 11 branch, 11.1 may come sooner than with previous major releases *** FreeBSD Core Election (https://www.freebsd.org/internal/bylaws.html) It is time once again for the FreeBSD Core Election Application period begins: Wednesday, 18 May 2016 at 18:00:00 UTC Application period ends: Wednesday, 25 May 2016 at 18:00:00 UTC Voting begins: Wednesday, 25 May 2016 at 18:00:00 UTC Voting ends: Wednesday, 22 June 2016 at 18:00:00 UTC Results announced Wednesday, 29 June 2016 New core team takes office: Wednesday, 6 July 2016 As of the time I was writing these notes, 3 hours before the application deadline, the candidates are: Allan Jude: Filling in the potholes Marcelo Araujo: We are not vampires, but we need new blood. Baptiste Daroussin (incumbent): Keep on improving Benedict Reuschling: Learn and Teach Benno Rice: Revitalising The Community Devin Teske: Here to help Ed Maste (incumbent): FreeBSD is people George V. Neville-Neil (incumbent): There is much to do… Hiroki Sato (incumbent): Keep up with our good community and technical strength John Baldwin: Ready to work Juli Mallett: Caring for community. Kris Moore: User-Focused Mathieu Arnold: Someone ask for fresh blood ? Ollivier Robert: Caring for the project and you, its developers The deadline for applications is around the time we finish recording the live show We welcome any of the candidates to schedule an interview in the next few weeks. We will make an attempt to hunt many of them down at BSDCan as well. *** Wayland/Weston with XWayland works on DragonFly (http://lists.dragonflybsd.org/pipermail/users/2016-May/249620.html) We haven’t talked a lot about Wayland on BSD recently (or much at all), but today we have a post from Peter to the dragonfly mailing list, detailing his experience with it. Specifically he talks about getting XWayland working, which provides the compat bits for native X applications to run on WayLand displays. So far on the working list of apps: “gtk3: gedit nautilus evince xfce4: - xfce4-terminal - atril firefox spyder scilab” A pretty impressive list, although he said “chrome” failed with a seg-fault This is something I’m personally interested in. Now with the newer DRM bits landing in FreeBSD, perhaps it’s time for some further looking into Wayland. Broadcom WiFi driver update (http://adrianchadd.blogspot.ca/2016/05/updating-broadcom-softmac-driver-bwn-or.html) In this blog post Adrian Chadd talks about his recent work on the bwn(4) driver for Broadcom WiFi chips This work has added support for a number of older 802.11g chips, including the one from 2009-era Macbooks Work is ongoing, and the hope is to add 802.11n and 5ghz support as well Adrian is mentoring a number of developers working on embedded or wifi related things, to try to increase the projects bandwidth in those areas If you are interested in driver development, or wifi internals, the blog post has lots of interesting details and covers the story of Adrian’s recent adventures in bringing the drivers up *** Beastie Bits The Design of the NetBSD I/O Subsystems (2002) (http://arxiv.org/abs/1605.05810) ZFS, BTRFS, XFS, EXT4 and LVM with KVM – a storage performance comparison (http://www.ilsistemista.net/index.php/virtualization/47-zfs-btrfs-xfs-ext4-and-lvm-with-kvm-a-storage-performance-comparison.html?print=true) Swift added to FreeBSD Ports (http://www.freshports.org/lang/swift/) misc@openbsd: 'NSA addition to ifconfig' (http://marc.info/?l=openbsd-misc&m=146391388912602&w=2) Papers We Love: Memory by the Slab: The Tale of Bonwick's Slab Allocator (http://paperswelove.org/2015/video/ryan-zezeski-memory-by-the-slab/) Feedback/Questions Lars - Poudriere (http://pastebin.com/HRRyfxev) Warren - .NET (http://pastebin.com/fESV1egk) Eddy - Sys Init (http://pastebin.com/kQecpA1X) Tim - ZFS Resources (http://pastebin.com/5096cGXr) Morgan - Ports and Kernel (http://pastebin.com/rYr1CDcV) ***
142: Diving for BSD Perls
This week on the show, we have all the latest news and stories! Plus an interview with BSD developer Alfred Perlstein, that you This episode was brought to you by Headlines The May issus of BSDMag is now out (https://bsdmag.org/download/reusing_openbsd/) GhostBSD Reusing OpenBSD's arc4random in multi-threaded user space programs Securing VPN's with GRE / Strongswan Installing XFCE 4.12 on NetBSD 7 Interview with Fernando Rodriguez, the co-founder of KeepCoding *** A rundown of the FPTW^XEXT.1 security reqiurement for General Purpose Operating Systems by the NSA (http://blog.acumensecurity.net/fpt_wx_ext-1-a-rundown/) NIST/NSA Validation Scheme Report (https://www.commoncriteriaportal.org/files/ppfiles/pp_os_v4.1-vr.pdf) The SFR or Security Functional Requirement requires that; "The OS shall prevent allocation of any memory region with both write and execute permissions except for [assignment: list of exceptions]." While nearly all operating systems currently support the use of the NX bit, or the equivalent on processors such as SPARC and ARM, and will correctly mark the stack as non-executable, the fact remains that this in and of itself is deemed insufficient by NIST and NSA. OpenBSD 5.8, FreeBSD, Solaris, RHEL, and most other Linux distro have failed. HardenedBSD passes all three tests out of the box. NetBSD will do so with a single sysctl tweak. Since they are using the PaX model, anything else using PaX, such as a grsecurity-enabled Linux distribution pass these assurance activities as well. OpenBSD 5.9 does not allow memory mapping due to W^X being enforced by the kernel, however the kernel will panic if there are any attempts to create such mappings. *** DistroWatch reviews new features in FreeBSD 10.3 (https://distrowatch.com/weekly.php?issue=20160516#freebsd) DistroWatch did a review of FreeBSD 10.3 They ran into a few problems, but hopefully those can be fixed An issue with beadm setting the canmount property incorrectly causing the ZFS BE menu to not work as expected should be resolved in the next version, thanks to a patch from kmoore The limitations of the Linux 64 support are what they are, CentOS 6 is still fairly popular with enterprise software, but hopefully some folks are interested in working on bringing the syscall emulation forward In a third issue, the reviewer seemed to have issues SSHing from inside the jail. This likely has to do with how they got a console in the jail. I remember having problems with this in the past, something about a secure console. *** BSD Unix: Power to the people, from the code (https://www.salon.com/2000/05/16/chapter_2_part_one/) Salon.com has a very long article, chronicling much of the history behind BSD UNIX. It starts with detailing the humble origins of BSD, starting with Bill Joy in the mid-70’s, and then goes through details on how it rapidly grew, and the influence that the University of Berkeley had on open-source. “But too much focus on Joy, a favorite target for business magazine hagiography, obscures the larger picture. Berkeley’s most important contribution was not software; it was the way Berkeley created software. At Berkeley, a small core group — never more than four people at any one time — coordinated the contributions of an ever-growing network of far-flung, mostly volunteer programmers into progressive releases of steadily improving software. In so doing, they codified a template for what is now referred to as the “open-source software development methodology.” Put more simply, the Berkeley hackers set up a system for creating free software.” The article goes on to talk about some of the back and forth between Linux and BSD, and why Linux has captured more of the market in recent years, but BSD is far from throwing in the towel. “BSD patriots argue that the battle is far from over, that BSD is technically superior and will therefore win in the end. That’s for the future to determine. What’s indisputable is BSD’s contribution in the past. Even if, by 1975, Berkeley’s Free Speech Movement was a relic belonging to a fast-fading generation, on the fourth floor of Evans Hall, where Joy shared an office, the free-software movement was just beginning.” An excellent article (If a bit long), but well worth your time to understand the origins of what we consider modern day BSD, and how the University of Berkley helped shape it. *** iXsystems (http://ixsystems.com) #ServerEnvy: It's over 10,000 Terabytes! (https://www.ixsystems.com/blog/serverenvy-10000-terabytes/) *** Interview - Alfred Perlstein - alfred@freebsd.org (mailto:alfred@freebsd.org) / @splbio (https://twitter.com/splbio) Using BSD for projects *** News Roundup .NET framework ported to NetBSD (https://github.com/dotnet/coreclr/pull/4504/files) This pull request adds basic support for the .NET framework on NetBSD 7.x amd64 It includes documentation on how to get the .NET framework installed It uses pkgsrc to bootstrap the required tools pkgsrc-wip is used to get the actual .NET framework, as porting is still in progress The .NET Core-CLR is now available for: FreeBSD, Linux, NetBSD, and OS X *** OpenBSD SROP mitigation – call for testing (https://marc.info/?l=openbsd-tech&m=146281531025185&w=2) A new technique for exploiting flaws in applications and operating systems has been developed, called SROP “we describe Sigreturn Oriented Programming (SROP), a novel technique for exploits and backdoors in UNIX-like systems. Like return-oriented programming (ROP), sigreturn oriented programming constructs what is known as a ‘weird machine’ that can be programmed by attackers to change the behavior of a process. To program the machine, attackers set up fake signal frames and initiate returns from signals that the kernel never really delivered. This is possible, because UNIX stores signal frames on the process’ stack.” “Sigreturn oriented programming is interesting for attackers, OS developers and academics. For attackers, the technique is very versatile, with pre-conditions that are different from those of existing exploitation techniques like ROP. Moreover, unlike ROP, sigreturn oriented programming programs are portable. For OS developers, the technique presents a problem that has been present in one of the two main operating system families from its inception, while the fixes (which we also present) are non-trivial. From a more academic viewpoint, it is also interesting because we show that sigreturn oriented programming is Turing complete.” Paper describing SROP (http://www.cs.vu.nl/~herbertb/papers/srop_sp14.pdf) OpenBSD has developed a mitigation against SROP “Utilizing a trick from kbind(2), the kernel now only accepts signal returns from the PC address of the sigreturn(2) syscall in the signal trampoline. Since the signal trampoline page is randomized placed per process, it is only known by directly returning from a signal handler.” “As well, the sigcontext provided to sigreturn(2) now contains a magic cookie constructed from a per-process cookie XOR'd against the address of the signal context.” This is just a draft of the patch, not yet considered production quality *** Running Tor in a NetBSD rump unikernel (https://github.com/supradix/rumprun-packages/tree/33d9cc3a65a39e32b4bc8034c151a5d7e0b89f66/tor) We’ve talked about “rump” kernels before, and also Tor pretty frequently, but this new github project combines the two! Specifically, this set of Makefile and scripts will prep a system to run Tor via the Unikernel through Qemu. The script mainly describes how to do the initial setup on Linux, using iptables, but could easily be adapted to a BSD if somebody wants to do so. (Send them a pull request with the instructions!) All in all, this is a fascinating way to run a Tor node or relay, in the most minimal operating environment possible. *** An update on SSH protocol 1 ("we're most of the way towards fully deprecating SSH protocol 1" (http://lists.mindrot.org/pipermail/openssh-unix-dev/2016-May/035069.html) Damien Miller has given us an update on the status of the “SSH protocol 1”, and the current plans to deprecate it in an upcoming version of openssh. “We've had this old protocol in various stages of deprecation for almost 10 years and it has been compile-time disabled for about a year. Downstream vendors, to their credit, have included this change in recent OS releases by shipping OpenSSH packages that disable protocol 1 by default and/or offering separate, non-default packages to enable it. This seems to have proceeded far more smoothly than even my most optimistic hopes, so this gives us greater confidence that we can complete the removal of protocol 1 soon. We want to do this partly to hasten the demise of this cryptographic trainwreck, but also because doing so removes a lot of legacy code from OpenSSH that inflates our attack surface. Having it gone will make our jobs quite a bit easier as we maintain and refactor.” The current time-line looks like removing server-size protocol 1 support this August after OpenSSH 7.4 is released, leaving client-side disabled. Then a year from now (June 2017) all protocol 1 code will be removed. Beastie Bits Last day to get your BSDNow Shirts! Order now, wear at BSDCan! (https://teespring.com/bsdnow) Move local government (Austin TX) from Microsoft Windows (incl. Office) to Linux and/or PC-BSD (https://github.com/atxhack4change/2016-project-proposals/issues/15) Plan9 boot camp is back... and already at capacity. Another opportunity may come in September (http://lists.nycbug.org/pipermail/talk/2016-May/016642.html) Smaller is better - building an openbsd based router (https://functionallyparanoid.com/2016/04/22/smaller-is-better/) Baby Unix (https://i.redditmedia.com/KAjSscL9XOUdpIEWBQF1qi3QMr7zWgeETzQM6m3B4mY.jpg?w=1024&s=e8c08a7d4c4cea0256adb69b1e7c1887) Security Update for FreeBSD (https://security.freebsd.org/advisories/FreeBSD-SA-16:19.sendmsg.asc) & Another security update for FreeBSD (https://security.freebsd.org/advisories/FreeBSD-SA-16:18.atkbd.asc) Feedback/Questions Eric - The iX experience (http://pastebin.com/ZknTuKGv) Mike - Building Ports (http://pastebin.com/M760ZmHQ) David - ZFS Backups (http://pastebin.com/Pi0AFghV) James - BSD VPS (http://pastebin.com/EQ7envez) Rich - ZFS Followup (http://pastebin.com/p0HPDisH) ***
141: BSD Likes Ike!
This week on the show, we have all the latest news and stories! Plus we’ll be hearing more about OpnSense from the man himself, Ike! This episode was brought to you by Headlines Regarding Embargoes (http://www.tedunangst.com/flak/post/regarding-embargoes) Our buddy TedU has a great thought piece today on the idea of “embargoes” for security advisories. This all stemmed from a recent incident with LibreSSL patches from embargoed OpenSSL vulns, that accidentally got committed too early. Ted makes a pretty good case on the difficulties of having embargos, and maybe the reason there shouldn’t be. Couple of quotes to give you a taste: “There are several difficulties maintaining embargoes. Keeping secrets is against human nature. I don’t want to be the one who leaks, but if I see something that looks like the secret is out, it’s a relief to be able to speak freely. There is a bias towards recognizing such signs where they may not really exist. (Exacerbated by broad embargoes where some parts leak but other parts don’t. It’s actually very hard to tell what’s not publicly known when you know everything.) The most thorough embargo and release timeline reconstruction is the heartbleed timeline. It’s another great case study. Who exactly decided who were the haves and have nots? Was it determined by who needed to know or who you needed to know? Eventually the dam started to crack.” “When Cloudflare brags that they get advance notice of vulnerabilities, attracting more customers, and therefore requiring even more early access, how are smaller players to compete? What happens if you’re not big enough to prenotify? Sometimes vulnerabilities are announced unplanned. Zero day cyber missiles are part of our reality, which means end users don’t really have the luxury of only patching on Tuesday. They need to apply patches when they appear. If applying patches at inconvenient times is a problem, make it not a problem. Not really a gripe about embargoes per se, but the scheduled timing of coordinated release at the end of the embargo is catering to a problem that shouldn’t exist.” I will admit that CloudFlare bragging around Heartbleed was upsetting The biggest issue here is the difficulty with coordinating so many open source projects, which are often done by volunteers, in different countries and time zones The other issue is determining when the secret is “out of the bag” *** MAJOR ABI BREAK: csu, ld.so, libc, libpthread update (http://www.openbsd.org/faq/current.html#r20160507) OpenBSD warns those following the -current (development) branch to be careful as they upgrade because of a major ABI break that will result in applications not working “Handling of single-threaded programs is now closer to multi-threaded, with ld.so and libc.a doing thread information base (TIB) allocation. Threaded programs from before the 2016/03/19 csu and ld.so update will no longer run. An updated ld.so must be built and installed before running make build.” A special note for those on PowerPC: “PowerPC has been updated to offset the TIB from the hardware register. As a result, all threaded programs are broken until they have been rebuilt with the new libc and libpthread. perl must be built after building the libraries and before building the rest of base.” “The definitions of environ and __progname for dynamically linked programs have been moved from the C startup code to ld.so(1). An updated ld.so must be built and installed before running make build.” The link provides instructions on how to update your system properly *** How to install FreeBSD 10.3 on VMWare Workstation 12 Pro (http://random-notes-of-a-sysadmin.blogspot.be/2016/04/howto-install-freebsd-103-on-vmware.html) This tutorial starts at the very basics, running through the FreeBSD installer But then it goes on to configuring the machine specifically for VMWare After the system has been booted, the tutorial walks through installing the VMWare tools Then networking is configured in both VMWare and FreeBSD A small hack is required to make the VMWare tools startup script wait until the network is up A very nice tutorial for people using VMWare I am working on a patch to bsdinstall to ensure that the swap partition is put before the main partition, so it can more easily be resized if you later decide you need more space in your VM the camcontrol reprobe subcommand has been added (https://svnweb.freebsd.org/base?view=revision&revision=299371), “This makes it possible to manually force updating capacity data after the disk got resized. Without it it might be necessary to reboot before FreeBSD notices updated disk size under eg VMWare.” *** BSD Router project releases v1.59 (https://sourceforge.net/projects/bsdrp/files/BSD_Router_Project/1.59/) We’ve talked about the BSD Router project a bit in the past, but today we have a brand new release to bring to you. For those who don’t remember, the BSDrp is a router aimed at replacing more of your big-commercial type systems. First up in the new hotness, we have it based upon recently released FreeBSD 10.3! In addition, there is a new package: New package: mlvpn (aggregated network links in order to benefit from the bandwidth of multiple links) Other packages have gotten a bump with this release as well: bsnmp-ucd to 0.4.2 dma to 0.11 dmidecode to 3.0 exabgp to 3.4.15 iperf3 to 3.1.2 monit to 5.17 mpd5 to 5.8 openvpn to 2.3.10 python to 2.7.11 quagga to 1.0.20160315 strongswan to 5.4.0 What are you waiting for? Amd64 and i386 images are ready for you to download now. Interview - Isaac (.Ike) Levy - See Ike again at SEMIBug in Troy, Michigan on May 17th (http://semibug.org/) *** News Roundup Tredly - Prebuilt containers on FreeBSD (https://github.com/tredly/) Discussion regarding its GPLv3 licensing (https://www.reddit.com/r/freebsd/comments/4gggw8/introducing_tredly_containers_for_unix_freebsd/) A new “container” solution called “Trendly” has started making some news around various tech sites. In particular, this new project uses FreeBSD as its base OS and jail functionality in the backend. Their solution seems based around the idea of shipping containers as manifests, such as lists of packages to install and configuration knobs. The project is still rather new, and we’ll be keeping an eye on it for the future. One notable change already though, it was (for some reason) released under GPLv3. Understandably this caused quite a ruckus with various folks in the community, since it’s built specifically on BSD. Since this, the code has been re-licensed as MIT, which is far more in the spirit of a traditional BSD license. *** NVMe driver added to NetBSD - ported from OpenBSD (https://www.netbsd.org/changes/changes-8.0.html#nvme%284%29) NetBSD has gained support for Non-Volatile Memory Express, the new standard for PCIe attached Flash Memory The change of interface from SATA to NVMe offers a number of advantages, mostly, it doesn’t require the device to pretend to be a spinning disk One of the biggest advantages is that it supports completing multiple operations at once, with the Intel hardware I have tested, 63 I/Os can happen concurrently, so a very large queue depth is required to keep the device busy. The 64th I/O channel is reserved for administrative commands, to keep them from being delayed by the large queue depth The device I tested could read at 3800 MB/s, and write 1700MB/s, something that wouldn’t be possible with a normal SSD It is interesting that NetBSD took the NVMe support from OpenBSD, whereas the FreeBSD implementation was contributed directly by Intel This may have to do with that fact that OpenBSD’s device model is closer to that of NetBSD Commit Log (http://mail-index.netbsd.org/source-changes/2016/05/01/msg074367.html) *** New BSDNow T-Shirts (https://teespring.com/bsdnow) By popular demand, we have created a more subtle BSDNow shirt Featuring only the smallish BSDNow logo over the left breast Available in a number of styles (T-Shirt, Women’s T-Shirt, Long Sleeve, and Hoodie) as well as a number of colours: Black, Blue, Grey, and White The hope is that enough orders come though so we can get them shipped in and your sweaty little hands in time for BSDCan. (I’ll be wearing mine, will you B...SD?) If you still want one of our now-famous “The Usual BSD’s” t-shirts, you can also indicate your interest here, and once 10 or more shirts are ordered, a reprint will happen automatically (https://teespring.com/bsd105) *** PC-BSD 11-CURRENT with Package Base (http://lists.pcbsd.org/pipermail/testing/2016-May/010616.html) Looking for a way to play with the new FreeBSD base package system? This month’s PC-BSD -CURRENT image now used packages for base system installation, and is asking for testers to help find bugs. Known issues so far: setuid binaries (Fix in works) Missing tzone files Distrib packages If all that doesn’t scare you away, then give it a whirl! Upgrades for previous APRIL images are now online also. *** BeastieBits HardenedBSD + LibreSSL (https://hardenedbsd.org/article/shawn-webb/2016-05-05/libressl-hardenedbsd-base) Michael Dexter's talk at LFNW 2016 is the 2nd highest youtube views from this years conference (https://www.youtube.com/watch?v=6k1Mf0c6YW8) Why OpenBSD is important to me (http://ggr.com/why-openbsd-is-important-to-me.html) Study of nginx-1.9.12 performance/latency on DragonFlyBSD-g67a73 (http://lists.dragonflybsd.org/pipermail/users/2016-May/249581.html) Running FreeBSD / OpenBSD / NetBSD as a virtualised guest on Online.net (https://www.geeklan.co.uk/?p=2109) The interesting story of how IllumOS syscalls work (http://zinascii.com/2016/the-illumos-syscall-handler.html) The BeaST is the FreeBSD based dual-controller reliable storage system concept with aim to implement ZFS and in-memory cache. (https://mezzantrop.wordpress.com/portfolio/the-beast/) Francois Tigeot updates the drm/i915 driver to match what’s in Linux kernel 4.3 (http://lists.dragonflybsd.org/pipermail/commits/2016-May/500352.html) FreeBSD is working on the update to Linux Kernel 4.6, we may finally get ahead of Dragonfly! (https://twitter.com/ed_maste/status/730450314889924608) Feedback/Questions Oskar - Torrent Jail (http://pastebin.com/RT7tVtQ7) Shane - ZFS Delete (http://pastebin.com/VkpMeims) Adam - Zimbra Port (http://pastebin.com/MmQ00Sv1) Ray - PC-BSD - FrameBuffer (http://pastebin.com/Xx9TkX7A) Richard - ZFS Backups (http://pastebin.com/ncYxqpg3) ***
140: Tracing it back to BSD
This week on BSDNow, Allan is back in down from Europe! We’ll get to hear some of his wrap-up and get caught up on the latest BSD This episode was brought to you by Headlines FreeBSD Quarterly Report (http://www.freebsd.org/news/status/report-2016-01-2016-03.html) This quarterly status report starts with a rather interesting introduction by Warren Block ASLR Porting CEPH to FreeBSD RCTL I/O Rate Limiting The Graphics Stack on FreeBSD (Haswell is in, work is progressing on the next update) CAM I/O Scheduler NFS Server updates, working around the 16 group limit, and implementing pNFS, allowing NFS to scale beyond a single server Static Analysis of the FreeBSD Kernel with PVS Studio PCI-express HotPlug GitLab Port committed! WITHFASTDEPEND and other improvements to the FreeBSD build system Lots of other interesting stuff *** A Prog By Any Other Name (http://www.tedunangst.com/flak/post/a-prog-by-any-other-name) Ted Unangst looks at what goes into the name of a program “Sometimes two similar programs are really the same program with two names. For example, grep and egrep are two commands that perform very similar functions and are therefore implemented as a single program. Running ls -i and observing the inode number of each file will reveal that there is only one file. Calling the program egrep is a shorthand for -E and does the same thing.” So BSD provides __progname in libc, so a program can tell what its name is But, what if it has more than one name? “In fact, every program has three names: its name in the filesystem, the name it has been invoked with, and whatever it believes its own name to be.” Of course it is not that easy. “there’s another set of choices for each name, the full path and the basename” “It’s even possible on some systems for argv[0] to be NULL.” He then goes on to rename doas (the OpenBSD light replacement for sudo) to banana and discuss what happens “On that note, another possible bug is to realize that syslog by default uses progname. A user may be able to evade log monitoring by invoking doas with a different name. (Just fixed.)” Another interesting article from our friend Ted *** FreeBSD (https://summerofcode.withgoogle.com/organizations/4892834293350400/) and NetBSD (https://summerofcode.withgoogle.com/organizations/6246531984261120/) Google Summer of Code projects have been announced Some FreeBSD highlights: Add SCSI passthrough to CTL (share an optical drive via iSCSI) Add USB target mode driver based on CTL (share a USB device via iSCSI) API to link created /dev entries to sysctl nodes Implement Ethernet Ring Protection Switching (ERPS) HD Audio device model in userspace for bhyve Some NetBSD highlights: Implement Ext4fs support in ReadOnly mode NPF and blacklistd web interface Port U-Boot so it can be compiled on NetBSD Split debug symbols for pkgsrc builds *** libressl - more vague priomises (http://www.tedunangst.com/flak/post/libressl-more-vague-promises) We haven’t had a Ted U article on the show as of late, however this week we get several! In his next entry “LibreSSL, more vague promises” He then goes into some detail on what has happened with LibreSSL in the past while, as well as future plans going forward. “With an eye to the future, what new promises can we make? Some time ago I joked that we only promised to make a better TLS implementation, not a better TLS. Remains true, but fortunately there are people working on that, too. TLS 1.3 support is on the short term watchlist. The good news is we may be ahead of the game, having already removed compression. How much more work can there be?” “LibreSSL integrated the draft chacha20-poly1305 construction from BoringSSL. The IETF has since standardized a slightly different version because if it were the same it wouldn’t be different. Support for standard variant, and the beginning of deprecation for the existing code, should be landing very shortly. Incidentally, some people got bent out of shape because shipping chacha20 meant exposing non IANA approved numbers to Internet. No promises that won’t happen again.” *** Interview - Samy Al Bahra - @0xF390 (https://twitter.com/0xF390) Backtrace *** News Roundup systrace(1) is removed for OpenBSD 6.0 (http://marc.info/?l=openbsd-cvs&m=146161167911029&w=2) OpenBSD has removed systrace, an older mechanism for limiting what syscalls an application can make It is mostly replaced by the pledge() system OpenBSD was the first implementation, most others have been unmaintained for some time The last reported Linux version was for kernel 2.6.1 NetBSD removed systrace in 2007 *** pfSense Video Series: Comprehensive Guide To pfSense 2.3 (https://www.youtube.com/playlist?list=PLE726R7YUJTePGvo0Zga2juUBxxFTH4Bk) A series of videos (11 so far), about pfSense Covers Why you would use it, how to pick your hardware, and installation Then the series covers some networking basics, to make sure you are up to speed before configuring your pfSense Then a comprehensive tour of the WebUI Then goes on to cover graphing, backing up and restoring configuration There are also videos on running DHCP, NTP, and DNS servers *** DuckDuckGo announces its 2016 FOSS Donations (https://duck.co/blog/post/303/2016-foss-donations-announcement) The theme is “raising the standard of trust online” Supported projects include: OpenBSD Foundation announces DuckDuckGo as a Gold Sponsor (http://undeadly.org/cgi?action=article&sid=20160503085227&mode=expanded) the Freedom of the Press Foundation for SecureDrop the Freenet Project the CrypTech Project the Tor Project Fight for the Future for Save Security Open Source Technology Improvement Fund for VeraCrypt (based on TrueCrypt) Riseup Labs for LEAP (LEAP Encryption Access Project) GPGTools for GPGMail *** Larry the BSD Guy hangs up his hat at FOSS Force (http://fossforce.com/2016/04/bsd-linuxfest-northwest/) After 15 years, Larry the BSD Guy has decided to hang it up, and walk into the sunset! (Figuratively of course) After wrapping up coverage of recent LinuxFest NorthWest (Which he didn’t attend), Larry has decided it’s time for a change and is giving up his column over at FOSS Force, as well as stepping away from all things technical. His last write-up is a good one, and he has some nice plugs for both Dru Lavigne and Michael Dexter of the BSD community. He will be missed, but we wish him all the luck with the future! He also puts out the plug that FOSS Force will be needing a new columnist in the near future, so if you are interested please let them know! *** Beastie Bits If you sponsored “FreeBSD Mastery: Advanced ZFS”, check your mail box (http://blather.michaelwlucas.com/archives/2648) pkg-1.7.0 is an order of magnitude slower than pkg-1.6.4 (https://marc.info/?l=freebsd-ports&m=146001143408868&w=2) -- Caused by a problem not in pkg LinuxFest Northwest 2016 Recap (https://www.ixsystems.com/blog/linuxfest-northwest-2016/) Dru Lavigne's 'Doc like an Egyption' talk from LFNW (https://www.linuxfestnorthwest.org/2016/sessions/doc-egyptian) Michael Dexters' 'Switching to BSD from Linux' talk from LFNW (https://www.linuxfestnorthwest.org/2016/sessions/devil-details-switching-bsd-linux) Michael Dexters' 'Secrets to enduring user groups' talk from LFNW (https://www.linuxfestnorthwest.org/2016/sessions/20-year-and-counting-secrets-enduring-user-groups) January issue of Freebsd Journal online for free (https://www.freebsdfoundation.org/journal/) Ghost BSD releases 10.3 Alpha1 for testing (http://ghostbsd.org/10.3_alpha1) EuroBSDcon 2016 - Call for Papers - Dealine: May 8th (https://www.freebsdnews.com/2016/04/15/eurobsdcon-2016-call-for-papers/) KnoxBUG Initial Meeting (http://www.knoxbug.org/content/knoxbug-maiden-voyage) Photos, slides, and videos from the Open Source Data Center Conference (https://www.netways.de/en/events_trainings/osdc/archive/osdc2016/) *** Feedback/Questions Mohammad - Replication (http://pastebin.com/KDnyWf6Y) John - Rolling new packages (http://pastebin.com/mAbRwbEF) Clint - Unicast (http://pastebin.com/BNa6pyir) Bill - GhostBSD (http://pastebin.com/KDjS2Hxa) Charles - BSD Videos (http://pastebin.com/ABUUtzWM) ***