This is a read-only archive. Find the latest Linux articles, documentation, and answers at the new!

Feature: System Administration

Easy automated editing of /etc/files with Augeas

By Ben Martin on June 12, 2008 (4:00:00 PM)

Share    Print    Comments   

The days of parsing configuration files with awk and making quick changes to configuration files with ad-hoc scripts may finally be at an end. With Augeas you can forget about the parsing and focus completely on what settings must be changed. So if the configuration file moves a piece of data to the fourth column, you don't need to care; Augeas will still show it to you as it did before.

Augeas lets you edit your configuration files from the command line, or from a program you are writing using the native C API or the bindings for Python, Ruby, or OCaml. Augeas is available in the standard Fedora 9 repositories and as a 1-Click install for openSUSE 10.3, but is not packaged for Ubuntu. I compiled it from source on a 64-bit Fedora 8 machine using Augeas version 0.1.1. Installation follows the normal ./configure; make; sudo make install procedure.

If you have executed the ./configure steps above (and as is indicated in the README) then your Augeas setup will probably be useless at first because the files that Augeas uses to recognize configuration files are installed in /usr/local but sought from /usr. The files that describe how to recognize a configuration file are called lens files -- more on them in a moment. You can fix this installation issue by exporting the correct path in AUGEAS_LENS_LIB as shown below. Notice that the first ls command does not show any files directory in its output.

$ augtool augtool> ls Not enough arguments for ls augtool> ls / augeas/ = (none) augtool> $ export AUGEAS_LENS_LIB=/usr/local/share/augeas/lenses $ augtool augtool> ls / augeas/ = (none) files/ = (none)

Augeas exposes your configuration files using what can be thought of as two virtual filesystems. The /files path exposes the configuration files, while /augeas exposes metadata. For example, your /etc/hosts file should appear as /files/etc/hosts though Augeas.

Augeas includes support for the aliases, aptsources, fstab, grub, hosts, ifcfg, inittab, pam, sshd, sysconfig, xinetd and yum configuration file formats. You can also add support for editing your own configuration files with Augeas using what it calls lenses. If you are already familiar with text parsing and Extended Backus-Naur Form (EBNF) you will be able to create an Augeas lens for your configuration file without much trouble. If you are not already familiar with EBNF, read up on it and some introductory material on text parsing and formal grammars before trying to create your own lens files. One of the main differences between traditional EBNF definitions and writing a lens for Augeas is that a lens has to be able to be applied to both parse a text file into an abstract syntax tree and also to convert a tree into a plain text file again.

The download page for Augeas includes a warning about Augeas being a prototype and cautions that you may end up with mangled configuration files using it. It offers a "backup" option to the command-line tool, which will preserve your old configuration file when you make changes. It also lets you run Augeas with a system root directory that is not "/". This lets you edit a copy of your configuration file in, say, /tmp/augeas-root/etc/hosts, as a regular user and check that the changes are in line with what they should be before you copy that file into /etc/hosts.

The below commands set up a personal copy of /etc for playing around. The find command makes a tarball of your configuration files from /etc, explicitly excluding the gconf and selinux directories, because they can be huge and are not interesting for quickly testing Augeas.

export AUGEAS_ROOT=/tmp/augeas-`id -u`-root rm -rf $AUGEAS_ROOT mkdir $AUGEAS_ROOT find /etc \( -path /etc/gconf -o -path /etc/selinux \) \ -prune -o -type f \ -print0 | xargs -0 tar cf $AUGEAS_ROOT/etc.tar tar -C $AUGEAS_ROOT -xf $AUGEAS_ROOT/etc.tar sudo chown -R $USER $AUGEAS_ROOT augtool -b

I discovered at this stage that although the /etc/hosts file is used in the Tour on the Augeas Web site, my hosts file did not show up. It turns out that Augeas did not like my hosts file containing blank lines. Once I removed all of them, I could see my /etc/hosts file though Augeas as shown below.

$ augtool augtool> ls /files/etc sysconfig/ = (none) ... hosts/ = (none) inittab/ = (none) ... augtool> ls /files/etc/hosts 1/ = (none) 2/ = (none) ... augtool> ls /files/etc/hosts/1 ipaddr = canonical = localhost.localdomain alias[1] = localhost alias[2] = lincomvm

As the above ls command for the localhost definition shows, if there can be more than one entry for an item, they are indexed with [n] where n is a number. Using the 1,2...n syntax for subdirectories that show the information for each definition in the hosts file might seem to make it hard to check whether there is already an entry for a particular IP address, such as The paths used with Augeas support wildcards and some other things from XPath syntax. The first command shown below will find if there is a definition already for in /etc/hosts. The second command shows the details for this entry. I then create and set an alias for the host directly by using alias[1] in the set command.

The documentation makes reference to the last() function returning the index of the final node. Unfortunately Augeas doesn't yet allow for arithmetic expressions to be used at this location, so my attempt to add a second alias using last()+1 as the index fails. The simple workaround is to first add a new node using the ins command, and then set it using last() to get the index of the last node, which is the one you just added. The last command checks whether an alias exists by using the match command and no indexing at all on the alias path to check against all alias nodes. Finally, I use the save command to make the changes permanent.

augtool> match /files/etc/hosts/*/ipaddr /files/etc/hosts/10/ipaddr augtool> print /files/etc/hosts/10 /files/etc/hosts/10 /files/etc/hosts/10/ipaddr = "" /files/etc/hosts/10/canonical = "" augtool> set /files/etc/hosts/10/alias[1] "l.c.alias" augtool> print /files/etc/hosts/10 /files/etc/hosts/10 /files/etc/hosts/10/ipaddr = "" /files/etc/hosts/10/canonical = "" /files/etc/hosts/10/alias = "l.c.alias" augtool> set /files/etc/hosts/10/alias[last()+1] "l.c2.alias" Failed augtool> ins alias after /files/etc/hosts/10/alias[last()] augtool> set /files/etc/hosts/10/alias[last()] "l.c2.alias" augtool> print /files/etc/hosts/10 /files/etc/hosts/10 /files/etc/hosts/10/ipaddr = "" /files/etc/hosts/10/canonical = "" /files/etc/hosts/10/alias[1] = "l.c.alias" /files/etc/hosts/10/alias[2] = "l.c2.alias" augtool> match /files/etc/hosts/10/alias l.c2.alias /files/etc/hosts/10/alias[2] augtool> save

Augeas removes a lot of the pain of updating configuration files from scripts. Of course you can always use Perl, awk, grep, wc, and cat to append a new line to a configuration file only if it does not already exist. But as the editing you perform on the configuration file becomes more complex, your script has to jump through more and more hoops to get the edit just right and then make it robust against configuration files that do not appear exactly like the ones you have seen.

While Augeas is a young project, and little things like the blank lines in a /etc/hosts file still seem to throw it off at the moment, it can already make configuration file edits a lot simpler to perform. What's more, once an Augeas lens is updated, then parsing issues like the blank line issue for the hosts file can be resolved in one place for everybody.

Ben Martin has been working on filesystems for more than 10 years. He completed his Ph.D. and now offers consulting services focused on libferris, filesystems, and search solutions.

Share    Print    Comments   


on Easy automated editing of /etc/files with Augeas

Note: Comments are owned by the poster. We are not responsible for their content.

Easy automated editing of /etc/files with Augeas

Posted by: Anonymous [ip:] on June 12, 2008 04:53 PM
Augeas also has python and ruby bindings. It is super simple to do very complex things with it.


Easy automated editing of /etc/files with Augeas

Posted by: Anonymous [ip:] on June 12, 2008 07:19 PM
Looks more like something from the Rube Goldberg regions of Comp Sci than a useful tool to me.

vi /etc/something.conf is fine for a one time change.

cp /etc/something_scenario_1.conf /etc/something.conf and cp /etc/something_scenario_2.conf /etc/something.conf is fine for a periodic change.

Have a few something.confs to change - as in you are using your laptop at work and then using your laptop at home, well there is nothing wrong with wrapping a group of something.conf changes in a shell script.

Neither the article nor the "tool's" web page make it clear to me what is gained by adding (IMO) a complex, and apparently needless layer of abstraction to a simple and straight forward issue.


Re: Easy automated editing of /etc/files with Augeas

Posted by: Anonymous [ip:] on June 12, 2008 08:12 PM
It's good for automating changes. Say you need to add/change/delete an entry to a config file on a pool of machines, the files contain different per-machine customizations, and some of the files already contain an incorrect but matching entry. It sounds like this would let you avoid having to write (and test!) a quick-n-dirty script in your favorite language. I like the idea of one tool that already understands lots of different config file formats, although I'll admit that the manipulation commands seem arcane.

I wonder if you can use the Perl/Python API as a way to load config files into a program? I've had config files that started out as .INI files, and had to change to something more complicated as requirments changed. This sounds like a tool that would let one program handle these situations.


Re(1): Easy automated editing of /etc/files with Augeas

Posted by: Anonymous [ip: unknown] on June 12, 2008 11:48 PM
From what I can see, this tool looks an awful lot like a poor mans configuration database or directory. The problem I would see is that if you really were responsible for managing a large number of machines this, much like an ersatz Windows registry, would lock you into an not-so-scalable interim solution when perhaps you would be better off making a slightly larger leap to something more robust


Well, maybe this does go the right way.

Posted by: Anonymous [ip:] on June 13, 2008 04:32 PM

Of course. A framework like this could make config handling and migration easier and transparent. For scripts and users alike, developers like users and desktop tools.


This story has been archived. Comments can no longer be posted.

Tableless layout Validate XHTML 1.0 Strict Validate CSS Powered by Xaraya