This is a read-only archive. Find the latest Linux articles, documentation, and answers at the new Linux.com!

Linux.com

Feature: Gentoo

Gentoo Portage secrets

By Aleksey Alekseyev on August 11, 2006 (8:00:00 AM)

Share    Print    Comments   

Gentoo Linux is perhaps the most-used source-based Linux distribution. One secret to its success is the powerful and handy Portage package management system. While Gentoo comes with extensive documentation covering most aspects of using Portage, the techniques described in Gentoo's handbook and other documentation are not always the most effective ones. Here are some insider tips that can greatly increase your productivity.

Search faster

Before you install a package, you usually look for it via Portage's search capabilities. Portage's emerge utility has --search and --searchdesc options, but using them is not enjoyable, because they take a long time to run. That's why we've seen the emergence of third-party search front ends for Portage, such as esearch and eix. Their common idea is to use their own search indexes to speed up searches. When using either utility, you have to rebuild the index after updating the Portage tree, and after installing and uninstalling software.

Of the two, eix works faster and has more capabilities. You can get information on the utility from its man page or by invoking eix --help. To use eix to search for a package whose name contains foo, simply invoke eix foo.

Eix is a very flexible tool. It can give you more information on packages than esearch or emerge -s. It can search through different fields (e.g. package name, category, or description), it can search for regular expressions or wildcard patterns, or do fuzzy searches, and its output can be configured for use in scripts.

Optimizing traffic usage

Updating your software can take a lot of network bandwidth. There are tools that help you decrease Portage's appetite. The most effective and well-known such tool is Deltup, which allows you to download deltas, or the differences between new and old versions of package source. That approach can save you up to 90% of the download size. The procedure for installing deltup is described in the Gentoo wiki.

Using Deltup has its drawbacks, however. Deltup will not resume fetching a delta if you lose a connection. Sometimes, the server generating the deltas is overloaded, and you must wait for a long time for it to generate your delta. Sometimes Deltup gets confused about which versions of packages you have. And, of course, generating a new package takes some CPU time. Still, Deltup should not break your system, so you should not be afraid to give it a try.

KDE maintainers generate the deltas for their packages themselves. This HOWTO describes using these deltas in Gentoo. There was a period when Portage supported a kdexdeltas USE flag, so you did not need to patch your sources manually, but after KDE 3.5 this functionality was not updated, so the kdexdeltas flag will not give you any advantage these days. Note that if you receive errors that report "digest verification failed" when using manually updated KDE sources, you can still install new KDE packages by using the emerge --digest command.

You can use the power of deltas not only when updating source packages, but also when updating your Portage tree. Emerge-delta-webrsync allows you to download daily patches to the Portage tree, which is less traffic-consuming than using emerge --sync. Just invoke emerge emerge-delta-webrsync, and next time you need to sync your Portage tree, run emerge-delta-webrsync

The Gentoo wiki provides more tips for users with poor Internet connections.

A new feature that came in Portage 2.1 can also help speed downloads. If you add the string parallel-fetch to the FEATURES variable in /etc/make.conf, emerge will download some packages' source code while compiling others that are in the queue. This reduces installation time, especially if you emerge many packages.

Faster package compilation

Every time you install or update some software under Gentoo, you need to wait until its source is compiled. Portage supports a set of tools that try to decrease the compiling time of your packages.

One of the ways to speed up compilation on a slow machine is to distribute the compiling task to another host. This is what Distcc aims to do. You can even use Windows boxes to assist in the task.

Ccache makes it possible to speed up compilation of the same code by caching compilation results. Confcache is a similar tool for caching results of test configuration scripts (./configure). It appeared at the same time as Portage 2.1, and still lacks full documentation, but it has been mentioned in the Gentoo Linux Newsletter. After spending some time in the unstable branch, it is now hardmasked, which means that confcache has some unresolved issues; if you try using the current confcache version (0.4.2) you will sometimes need to install some packages with the confcache feature turned off, or clean the confcache cache (rm -rf /var/tmp/confcache).

All the tools mentioned in this section can be used for any compilation tasks. See the appropriate documentation for Distcc, ccache, and confcache (coming later).

Managing logs and configuration files

Another new feature in Portage 2.1 is the new logging framework. Many packages show notices while you emerge them, but you may not see them if you do not watch the emerge process. Now you just need to turn on elog, by adding these lines to /etc/make.conf:
# This sets what to log
PORTAGE_ELOG_CLASSES="warn error log"
# And this is how to do it
PORTAGE_ELOG_SYSTEM="save"
and creating the /var/log/portage/elog directory. Now, each package's emerge log will be saved in a separate file in the specified directory. You can find more info on configuring elog by inspecting the /etc/make.conf.example file. There are already GTK and QT graphical front ends for viewing these logs.

After updating some packages you sometimes need to update their configuration files. The standard tool for this is etc-update, but there is a more handy and advanced utility -- dispatch-conf. It automatically updates configuration files that have easy changes to comments, or that have not been changed between updates. It also allows you to redo any changes you made to configuration files by placing them into a version control system.

Would you like to know more?

If these few brief tips have whetted your appetite, you can learn more about most aspects of Portage from its handbook, official documentation, and the Gentoo wiki. If you have any further questions, you may find an answer on the Gentoo forums.

Share    Print    Comments   

Comments

on Gentoo Portage secrets

Note: Comments are owned by the poster. We are not responsible for their content.

Gentoo HOWTOS

Posted by: Anonymous Coward on August 11, 2006 07:29 PM
<a href="http://linux.coconia.net/" title="coconia.net">http://linux.coconia.net/</a coconia.net> for info on Gentoo installs:

HOWTO: GENTOO 2006.0, GNOME, KDE AND XFCE IN LESS THAN 180 MINUTES.

Simply copies the installation to you hard-drive.

HOWTO: GENTOO 2005.1 INSTALLATION SCRIPT/WALK-THROUGH.

A more traditional Gentoo installation (still a good guide to many Gentoo 2006 architectures).

Find out more about it at: <a href="http://linux.coconia.net/" title="coconia.net">http://linux.coconia.net/</a coconia.net>

There is a new page on installing MPlayer/MEncoder (a movie/video Player and Encoder).

There is a page on Cloning Windows XP partitions.

There is a page on installing Windows on a spare partition on your harddrive.

There is also two HOWTOs on accessing and writing to Windows XP/2000 (formatted with the NTFS) from Linux.

Access to Windows by Captive-FUSE and NTFS-FUSE are compared.

There is also code for JAVASCRIPT MOVIES (and a few to watch).

Also has a section on how to REMIX YOUR 14 Debian CDs as 2 DVDs.

#

Gentoo HOWTOS

Posted by: Anonymous Coward on August 11, 2006 07:12 PM
<a href="http://linux.coconia.net/" title="coconia.net">http://linux.coconia.net/</a coconia.net> for info on Gentoo installs:

HOWTO: GENTOO 2006.0, GNOME, KDE AND XFCE IN LESS THAN 180 MINUTES.

Simply copies the installation to you hard-drive.

HOWTO: GENTOO 2005.1 INSTALLATION SCRIPT/WALK-THROUGH.

A more traditional Gentoo installation (still a good guide to many Gentoo 2006 architectures).

Find out more about it at: <a href="http://linux.coconia.net/" title="coconia.net">http://linux.coconia.net/</a coconia.net>

There is a new page on installing MPlayer/MEncoder (a movie/video Player and Encoder).

There is a page on Cloning Windows XP partitions.

There is a page on installing Windows on a spare partition on your harddrive.

There is also two HOWTOs on accessing and writing to Windows XP/2000 (formatted with the NTFS) from Linux.

Access to Windows by Captive-FUSE and NTFS-FUSE are compared.

There is also code for JAVASCRIPT MOVIES (and a few to watch).

Also has a section on how to REMIX YOUR 14 Debian CDs as 2 DVDs.

#

Re:Gentoo HOWTOS

Posted by: Anonymous Coward on August 11, 2006 11:37 PM
Your shift key seems to be stuck sometimes.

#

Re:Gentoo HOWTOS

Posted by: Anonymous Coward on August 12, 2006 05:04 AM
I can not help wonder what makes people use a source-only distribution like Gentoo and what kind of people they are.

You are a bad example.

#

Re:Gentoo HOWTOS

Posted by: Anonymous Coward on August 16, 2006 07:35 PM
People tend to assume that source-based distributions are complicated and slow to install, so they wonder who could ever use them.

My experience is that, nowadays, to setup a Gentoo box is not different from any other distro, because it installs precompiled (!) binary packages. The main difference is that, from then on, you can install packages with just one command, exactly like apt-get... BUT it will be extra-tailored to your environment with ONLY the features you really need. No more bloated Gnome libs when I just want Firefox and KDE (or viceversa), no more having to sigh because "my distro still doesn't support feature X or Z".

The compilation times, on a modern machine, are usually just a few minutes... and for very big packages like OpenOffice or Firefox, you still have the chance to download binaries.

Gentoo users are like "lazy" Slackware users, they like simplicity and control, but with ease... try installing HAL and dbus on Slackware, and you'll cry; try *removing* HAL and dbus from Ubuntu, and you'll cry as well. If you don't care about controlling and tweaking your environment, then Gentoo is not the distribution for you, but then maybe you don't really need Linux at all.

#

Re:Gentoo HOWTOS

Posted by: Anonymous Coward on August 31, 2006 11:03 PM
I have been a Gentoo user for the past 3+ years. I tried Ubuntu about 6 months ago. I had more issues with Ubuntu, so I went back to Gentoo.

Install times are not that long for the bulk of the packages. There are a few that take awhile, but they are well known. KDE, Gnome, Xorg, OpenOffice

With Ubuntu, I couldn't get a couple of packages that weren't in the repository to work. I tried the debian version of DVD Styler and had compatibility issues, I believe.

#

Re:Gentoo HOWTOS

Posted by: Anonymous Coward on August 14, 2006 02:05 PM
Gentoo it uses to much ram on emerge sync and compiling, gentoo users dont know how to use linux, they only knnow how to install it. a useless distro, slackware is the only decent distro out there.

#

Re:Gentoo HOWTOS

Posted by: Anonymous Coward on August 16, 2006 06:32 PM
That makes gentoo suitable for everyone from newbie to professional.

You are able to control every compile things if you want even<nobr> <wbr></nobr>./configure<nobr> <wbr></nobr>... you can edit ebuilds or write it one for some new<nobr> <wbr></nobr>.tar.gz sources and not need to worry if 'make uninstall' does not exists on Makefile.

Even only "Boot Process" is far much advance compared to other distributions. How you turn off apache, squid, nfs, everything except ssh in just one-command using slackware when your<nobr> <wbr></nobr>/var partition corrupt<nobr> <wbr></nobr>:P

#

I dumped Gentoo for Ubuntu

Posted by: Anonymous Coward on August 14, 2006 08:13 PM
I see no real reason for me to endure the pain of compiling from source! Despite having ccache and distcc, installing and updating packages are a freaking pain in the @ss. Ubuntu rocks. I regret not installing Ubuntu earlier.

#

Re:I dumped Gentoo for Ubuntu

Posted by: Anonymous Coward on August 16, 2006 01:33 AM
If you don't know the advantages of Portage, then Gentoo isn't distribution for you and I'm glad you've found a better distro for you.
And btw. you DON'T have to update every day and every package. It's your choice what and when you'll update. You can simply update only security issues and when you really want new version of any package..
Btw. nice reading - <a href="http://tinyurl.com/o2am9" title="tinyurl.com">http://tinyurl.com/o2am9</a tinyurl.com><nobr> <wbr></nobr>:-)

#

Re:I dumped Gentoo for Ubuntu

Posted by: Anonymous Coward on November 17, 2006 06:09 AM
But there are so few *real* advantages. Conditional compilation? Made necessary by lazy devs. Processor optimization? Rarely a real advantage. The advantages are far outweighed by the advantages, imho.

#

Re:I dumped Gentoo for Ubuntu

Posted by: Anonymous Coward on December 01, 2006 07:05 AM
Screw all that - Gentoo is more fun. If you don't think so, keep using Ubuntu, nobody really cares.

#

Kuroo

Posted by: Anonymous Coward on September 01, 2006 02:42 PM
To all those who prefer a gui over cli, try <a href="http://kuroo.org/" title="kuroo.org">kuroo</a kuroo.org> I've been using it for a few weeks and I'm loving it!

#

Gentoo Portage secrets

Posted by: Anonymous [ip: 12.205.238.162] on August 27, 2007 11:53 PM
you know what? i agree. ubuntu knocks the socks off gentoo. gentoo is a good idea. i am running right now a dual boot (actually ternary (?) boot) of winxp pro, gentoo, and ubuntu. i ran into just about every problem in the book installing gentoo and finally got it working when the login crashed. i installed ubuntu in my spare space and it's awesome. you *could* compile everything yourself but downloadable binaries are waaaaaaaaay faster than compiling everything. when i have programming homework to do and have to install a new dev package I don't always have five hours to sit waiting for compile. btw distcc+ccache+flag optimization barely make a noticable difference. i like gentoo, i just like ubuntu better.

#

This story has been archived. Comments can no longer be posted.



 
Tableless layout Validate XHTML 1.0 Strict Validate CSS Powered by Xaraya