This is a read-only archive. Find the latest Linux articles, documentation, and answers at the new Linux.com!

Linux.com

Feature: System Administration

Automated user management with Expect

By Sergio Gonzalez Duran on September 26, 2007 (9:00:00 AM)

Share    Print    Comments   

At the large school in Mexico where I'm employed as a system manager, I proposed (and got) a Linux server to replace an old Windows 2000 file server and domain controller for the alumni. I then was faced with the task of adding 3,000 users to this new CentOS 5 server. I wasn't about to add thousands of users and their passwords one by one to the new Samba primary domain controller (PDC) system. With a little help from OpenOffice.org Calc, a utility called Expect, and shell scripts, I automated the process.

I previously set up Samba as the PDC, added groups, and configured the firewall -- all the basics -- before I began the task of adding users. I started by creating a Calc worksheet, into which I placed the student ID, full name, and school section, in a format something like this:

A B C
1 Student ID Full Name School Section
2 12345 CARDENAS LAZARO primaria
3 12467 JUAREZ BENITO secundaria

The school section is the main group for each student, and they are also members of the domain controller's users and domain_users groups. Using simple concatenation operators (&), I put it all together in the spreadsheet's fourth column (="useradd -s /sbin/nologin -G users,domain_users,"&C2&" -c """&B2&""""&" "&A2). It ends up, for each row, something like this:

useradd -s /sbin/nologin -G users,domain_users,primaria -c "CARDENAS LAZARO" 12345

I use -s /sbin/nologin because no student should be able to talk to the Linux server directly. -G adds the users to their respective supplementary groups -- very important, because they need to be recognized by Samba as part of the domain -- and the school section group is an academic control for the computer teachers.

In addition to the real users, you have to add the machines too as users both for Linux and Samba. This command should do it:

useradd -d /dev/null -g 100 -s /bin/false -c "PC001" -M PC001$; smbpasswd -a -m PC001

The above creates a null acount (without login, home dir, etc.) with the machine name PC001$ as a Linux account, then adds it as a Samba machine account (-m).

After I created the rows in Calc, I copied the resulting column into a text editor and saved it as users.sh. I changed permissions to make it executable (chmod 700 users.sh) and ran the script (./users.sh).

The next part is the tricky one. The students are given their user names -- their student IDs -- and a password they could use to log in from their Windows XP machines, but in Samba, you first need to use the smbpasswd or pdbedit command to add the user, and it then prompts for the password. I needed a tool that allowed me to do that through a shell script and a user/password text file.

Automating with Expect

As its Web site says, with Expect "you will be able to automate tasks that you've never even thought of before." I used it to interactively add a user/password combination from a text file that feeds a smbpasswd script shell.

Since I was not going to use the old accounts, I was creating everything from scratch. Again using OpenOffice.org Calc, I entered something like this:

A B C D E F
1 Student ID CHAR 1 CHAR 2 CHAR 3 CHAR 4 USER / PASSWORD
2 12345 r h b e 12345 rhbe
3 12467 s e g o 12467 sego

The formula I used in B:E, =CHAR(RANDBETWEEN(97;122)), generates random ASCII characters from a to z. I use only lower-case characters, and actually eight characters for the real passwords. I concatenate everything in the last column, F, then copy it to a text file. I end up with a file with two columns, username and password, and one row for each student. This file, passwords.txt, is the one that feeds the next script, which I call adduserstosamba.sh:

while read row ; do 
   user=`echo $row | gawk '{ print $1 }'`
   pass=`echo $row | gawk '{ print $2 }'`
   adduserstosamba2.sh $user $pass 
done < passwords.txt

As you can see, the script reads the file with the user/password combination and breaks each row into two variables, $user and $pass, using the gawk command. It then passes the two variables as arguments to the next script, adduserstosamba2.sh, which is the Expect script, the one that does the magic:

#!/usr/bin/expect -f 

  set password [lindex $argv 1] 
  spawn smbpasswd -a [lindex $argv 0] 
  expect { 
        -re "password:" {sleep 1; send "$password\r"; exp_continue} 
        -re "password:" {sleep 1; send "$password\r";} 
}

The first line indicates that a shell script that uses Expect it is going to be executed. The -f flag prefaces a file from which commands are read. Expect receives the arguments from the first script ($user, $pass) in the argv variable which is invoked through [lindex $argv n]. Next, spawn is an Expect command that creates a new process, in this case running the program smbpasswd with the -a option to create a new Samba user. The argument for this option is the student ID that is cached with $argv 0, or the first argument ($user) passed through the first script.

Next, expect is invoked. It reads (-re) what the smbpasswd -a command send to the terminal; if you had run, for example, smbpasswd -a john, then the next line that pop ups on the screen should include the string password:, because smbpasswd is expecting user input for the password. The next set of commands are all in braces { }. Sleep 1 stops the execution for one second so the terminal is ready for the input. The send command sends the content of the $password variable along with a \r character (an Enter, just like a normal user would do). exp_continue indicates to Expect that the execution is not over, so it goes and does the same for the confirm password line. Now execution finishes, and so does the adduserstosamba2.sh script. adduserstosamba.sh reads the next line from passwords.txt, and the whole process is repeated again -- in my case 3,000 times.

In less than two hours, the whole script was done, and my 3,000 users were ready to log in to the brand new Linux box, although very few students realize what they are connecting to. That is really the beauty of Samba -- let people work smoothly without worrying what tools are working on the background.

I hope and "Expect" that this kind of automation will be useful for you.

Sergio Gonzalez Duran is a Linux administrator, systems developer, and network security counselor who also teaches Linux courses and publishes the Spanish-oriented Linux and open source Web site linuxtotal.com.mx.

Share    Print    Comments   

Comments

on Automated user management with Expect

Note: Comments are owned by the poster. We are not responsible for their content.

Automated user management with Expect

Posted by: Anonymous [ip: 87.126.20.126] on September 26, 2007 11:24 AM
Not that it distracts from the usefulness of the article, but in case you use smbpasswd, couldn't you have used the -s option of smbpasswd? Then expect wouldn't be necessary:

-s This option causes smbpasswd to be silent (i.e. not issue prompts) and to read its old and new passwords from standard input, rather than from /dev/tty (like the passwd(1) program does). This option is to aid people writing scripts to drive smbpasswd.

#

Automated user management with Expect

Posted by: Anonymous [ip: 196.209.166.78] on September 26, 2007 12:13 PM
Very useful, thank you!

#

Automated user management with Expect

Posted by: Anonymous [ip: 83.27.52.6] on September 26, 2007 01:08 PM
Wouldn't it be better to use something like LDAP to store so many accounts informations?

#

Re: Automated user management with Expect

Posted by: walt-sjc on September 27, 2007 01:14 AM
Personally, I find LDAP to be much more useful. I have everything authenticating against it - lots of different web tools (LDAP integrates into many CMS systems, wikis, etc.) plus proxies (squid), email, etc. Anything that doesn't support LDAP out of the box can usually be modified without too much work to support it.

Maintaining multiple password databases just seems so primitive now... Plus with LDAP, you can delegate admin access to subtrees and such, which you just can't do with simplistic flat user tables.

A quick little perl script imports / deletes users based on CSV files.

#

Automated user management with Expect

Posted by: Anonymous [ip: 206.248.137.79] on September 26, 2007 01:14 PM
Using a spreadsheet is fine, but don't generate passwords in the spreadsheet. This is a "smoking gun", and the "random" number generator in your spreadsheet is not very random.

Instead, you can use the "mkpasswd" command. And it combines nicely with the --stdin option on the passwd command. Here is the bits of the script I use to do it:

#!/bin/sh

USERID=$1

PASSWORD=`mkpasswd -s 0`
echo "$PASSWORD" | /usr/bin/passwd --stdin $USERID

/usr/sbin/sendmail -oi -t <<EOF
To: $USERID <$USERID@domain.com>
From: System Adminstrator <root@domain.com>
Subject: Your new password

An account has been created for you:
Username: $USERID
Password: $PASSWORD

EOF

#

Strong password generation

Posted by: Anonymous [ip: 169.233.25.226] on September 26, 2007 01:55 PM
<code>PASSWORD_LENGTH="10" && tr -cd [:graph:] < /dev/urandom | head -c $PASSWORD_LENGTH</code>

Note: I have only confirmed this works on GNU/Linux. OpenBSD's head utility does not support the -c option.



Linux.com developers, please use <a href="http://php.net/nl2br">nl2br</a>!

#

Re: Automated user management with Expect

Posted by: Anonymous [ip: 127.0.0.1] on September 27, 2007 03:05 AM
Hate to be nitpicky, but the passwords emitted by 'mkpasswd -s 0' are dangerously poor. That's just giving you the des hash of a null password with 12 bits of salt, or only 4096 possibilities.



$ [run mkpasswd -s 0 >> despass.txt a lotta times]
$ cat despass.txt | wc -l
118720
$ cat despass.txt | sort | uniq | wc -l
4095

#

Re(1): Automated user management with Expect

Posted by: Anonymous [ip: 206.248.137.79] on September 29, 2007 12:10 AM

I agree that "-s 0" leads to a smaller set of characters, and therefore weaker passwords. But dangerously poor? With -s0 you get passwords that are 9 characters long and each character has 26+26+10=62 possibilities. That works out to some 13*10^18 possible strings. If I run the shell commands like you suggested, I find that on a similar size of ~100,000 generated passwords, the sort/uniq tells me there are NO duplicates at all- I don't know how you came up with "only" 4095 unique values.



Maybe its the DES algorithm. Of course, nobody uses that - passwords have been hashed using md5 for many years now. The md5 hash you can see in /etc/shadow are much longer than the 9 character password. I would not expect any collisions in this process. If there are any, then md5 is a lousy algorithm. (Okay, it has been shown that md5 can be factored, and SHA is better, but this is still far from everyday hackable).



Am I missing something? For sure, using -s0 leads to weaker passwords, but consider the hassle: how many of your lusers can distinguish ' from `, or locate the ^ symbol on their keyboards. For me, having users complain that their "password does not work" on a regular basis is a big pain, and if it ends up in me resetting those "wacky" passwords to "regular" ones, then we might have well just used -s0 in the first place.


#

Automated user management with Expect

Posted by: FrdPrefct on September 26, 2007 01:50 PM
I would suggest staying away from expect to do this process. There are better, faster, and more secure methods to create this script. Try reading the man pages for each of the utilities used. Good start though!

...frd

#

Much easier with 'net rpc vampire'

Posted by: Anonymous [ip: 83.67.74.208] on September 26, 2007 03:15 PM
Set up the Samba server as a BDC, and run net rpc vampire to copy all the account information across.

It also keeps all the SIDs etc. the same, which is important if you have any security stuff setup.

#

Re: Much easier with 'net rpc vampire'

Posted by: Anonymous [ip: 74.94.87.209] on September 26, 2007 04:13 PM
Although something may have changed in the recent past to allow this, 'net rpc vampire' won't work with Windows 2000 SBE since it doesn't allow a BDC. I ran into this problem a few years back and had to do something very similar to what was described in this article.

#

Expect script could be improved

Posted by: Anonymous [ip: 199.172.169.7] on September 26, 2007 05:20 PM
It's great that it solved your problem, but this Expect script could be improved.
The option -re does not mean read, it means to treat the next thing as a Regular Expression when matching against the output from the spawned program. Since you're only matching a fixed string, -re is redundant here.
The second match line is also redundant, because the clauses within an expect statement will always be matched in order and since the patterns are the same the second will never be reached. The exp_continue means to restart the same expect statement again.
Also Expect is capable of doing the whole job, without needing a shell script or gawk. Here's an (untested) example:

#!/usr/bin/expect -f
foreach {name pass} [read [open passwords.txt]] {
spawn smbpasswd -a $name
expect {
"password:" {sleep 1; send "$password\r"; exp_continue}
}
}

For more docs see http://aspn.activestate.com/ASPN/docs/ActiveTcl/8.4/expect/expect.1.html

#

Re: Expect script could be improved

Posted by: Anonymous [ip: unknown] on September 26, 2007 05:45 PM
Very good and ilustrative article and even better feedback, like this improvment, this is what I was looking for. Thanks a lot both writers.

#

Automated user management with Expect

Posted by: Anonymous [ip: 66.92.218.124] on September 26, 2007 07:01 PM
Should also look at the Python native pexpect.
Pure Python, nice syntax: pexpect.sourceforge.net

#

Flame Rate

Posted by: Anonymous [ip: 130.76.32.145] on September 27, 2007 03:07 PM
At least the FLAME levels seem pretty low around here.

#

Automated user management with Expect

Posted by: Anonymous [ip: 203.199.26.34] on October 11, 2007 01:32 PM
Hi,

Iam a newbie to this Expect Utility,

I wanted to know if this utility will work in a senario where we Send command but we dont know what we are going to receive.
that means... i know the end result... but if that is not recived, then i want to take the data received and convert to binary.
I wanted to know more on how to use this.
pl help me.

Rgds
Queenie

#

Somewhat similar thing for people using mac

Posted by: Anonymous [ip: 66.92.90.34] on November 07, 2007 09:26 PM
From my description:
This script was written to add users in a batch to an Open Directory LDAP server on Max OSX 10.4.

It expects a list of people to add in the form of First Name [TAB] Last Name [TAB] password [TAB] age. You can modify this to suit your needs; I needed to set groups based on the user's age, but you can do it however you need.

I had to install gcc via the xtools package (available for free from ADC) in order to compile Text::Unidecode --

http://pastebin.ca/765139

#

This story has been archived. Comments can no longer be posted.



 
Tableless layout Validate XHTML 1.0 Strict Validate CSS Powered by Xaraya