Expect Tutorial

#Except

Expect is a tool primarily for automating interactive applications such as telnet, ftp, passwd, fsck, rlogin, tip, etc. Expect really makes this stuff  trivial. Expect is also useful for testing these same applications

Pros

Expect can be run at regular intervals through the use of cron to encapsulate system administration tasks. This works because Expect merely uses system administration tools already located on the host computer. No extra tools need to be learned. If the programmer has already learned Tcl, then migrating to Expect is a relatively easy transition. The same programming structures and syntax exist, but with additional features built in.

There is large support in the industry for using Expect for many in-house administration tasks. It is widely used by companies such as Silicon Graphics, IBM, HP, Sun, Xerox, Amdahl, Tektronix, AT&T, ComputerVision and the World Bank to run in-house automated testing for development projects, file transfers, account administration, and network testing. As a Tcl extension, expect uses the same syntax, which is more regular and uniform than languages such as bash, csh, and Perl. Expect goes to great lengths to abstract away the differences between terminal behaviour on  various platforms. Other programs similar to expect are usually lacking in this regard. Because it is a Tcl extension, the full facilities of Tcl are available for use in expect scripts. Additional packages and extensions for Tcl can easily be used to expand the features of the Expect script.

Cons

Expect cannot automate GUI based tools. This is generally only a problem on Windows where for many tasks a GUI based interface is the only option. In these situations, tools like AutoHotkey, AutoIt, or Winbatch can be used instead.

---

Get Started With Expect

The three commands send, expect, and spawn are the building power of Expect. The send command sends strings to a process, the expect command waits for strings from a process, and the spawn command starts a process.

---

The send Command

The send command takes a string as an argument and sends it to a process. For example:

send "hello world"

This sends the string "hello world" (without the quotes). If Expect is already interacting with a program, the string will be sent to that program. But initially, send will send to the standard output. Here is what happens when I type this to the Expect interpreter interactively:

% expect

expect1.1>send "hello world"

hello worldexpect1.2>exit

%

The send command does not format the string in any way, so after it is printed the next Expect prompt gets appended to it without any space. To make the prompt appear on a different line, put a newline character at the end of the string. A newline is represented by "\n". The exit command gets you out of the Expect interpreter.

expect1.1>send "hello world\n"

hello world

expect1.2>exit

%

If these commands are stored in a file, speak, the script can be executed from the UNIX command line:

% expect speak

hello world

To execute the file as just "speak" rather than "expect speak", insert the line "#!./expect -f" and do "chmod +x speak" . The name of the interpreter must appear after the characters #! in the first line. The ./expect is the path where Expect is to be found; in this case, it is in the current working directory.

% cat speak

#!./expect -f

send "hello world\n"

%

% chmod +x speak

% speak

hello world

---

The expect Command

The expect command waits for a response, usually from a process. expect can wait for a specific string or any string that matches a given pattern. Like send, the expect command initially waits for characters from the keyboard. To see how the expect command works, create a file response.exp that reads:

#!./expect -f

expect "hi\n"

send "hello there!\n"

When I make response.exp executable and run it, the interaction looks like this:

% chmod +x response.exp

% response.exp

hi

hello there!

If you get an error that goes like,couldn't read file " ": No such file or directory it may be because there are non-printable characters in your file. This is true if you do cut-and-paste from Netscape to your file. To solve this problem, try deleting trailing spaces at the end of each command line (even if there seems to be nothing there) in the script and follow the above steps again.

What Happens When Input Does Not Match

If expect reads characters that do not match the expected string, it continues waiting for more characters. If I had type hello instead of hi followed by a return, expect would continue to wait for "hi\n". Finding unexpected data in the input does not bother expect. It keeps looking until it finds something that matches. If no input is given, expect command eventually times out and returns. By default, after 10 seconds expect gives up waiting for input that matches the pattern. This default value can be changed by setting the variable timeout using the Tcl set command. For example, the following command sets the timeout to 60 seconds.

set timeout 60

A timeout of -1 signifies that expect should wait forever and a timeout of 0 indicates that expect should not wait at all.

Anchoring

To prevent expect from matching unexpected data, expect patterns can include regular expressions. The caret ^ is a special character that only matches the beginning of the input; it cannot skip over characters to find a valid match. For example, the pather ^hi matches if I enter "hiccup" but not if I enter "sushi" . The dollar sign ($) is another special character. It matches the end of the data. The pattern hi$ matches if I enter "sushi" but not if I enter "hiccup". And the pattern ^hi$ matches neither "sushi" nor "hiccup". It matches "hi" and nothing else.

Patterns that use ^ or $ are said to be anchored. When patterns are not anchored, patterns match beginning at the earliest possible position in the string. For more techniques on pattern matching, I suggest you buy the the book, Exploring Expect as well as Tcl and The Tk Toolkit.

Pattern-Action Pairs

Expect also allows the association between a command and a pattern. The association is made by listing the action (also known as command) immediately after the pattern in the expect command itself. Here is an example of pattern-action pairs:

 expect "hi"  { send "You said hi\n" } \

    "hello"   { send "Hello yourself\n" } \

    "bye"     { send "Good-bye cruel world\n" }

This command looks for "hi", "hello", and "bye". If any of the three

patterns are found, the action immediately following it gets executed.

If there is no match and the default timeout expires, expect stops

waiting and execution continues with the next command in the script.

---

The spawn Command

The spawn command starts another program. The first argument of the spawn command is the name of a program to start. The remaining arguments are passed to the program. For example:

spawn ftp ftp.uu.net

This command spawns an ftp process and ftp.uu.net is the argument to the ftp process.

---

Putting It All Together

#A simple example is a script that automates a telnet session:

# Assume $remote_server, $my_user_id, $my_password, and $my_command were read in earlier

# in the script.

# Open a telnet session to a remote server, and wait for a username prompt.

spawn telnet $remote_server

expect "username:"

# Send the username, and then wait for a password prompt.

send "$my_user_id\r"

expect "password:"

# Send the password, and then wait for a shell prompt.

send "$my_password\r"

expect "%"

# Send the prebuilt command, and then wait for another shell prompt.

send "$my_command\r"

expect "%"

# Capture the results of the command into a variable. This can be displayed, or written to disk.

set results $expect_out(buffer)

# Exit the telnet session, and wait for a special end-of-file character.

send "exit\r"

expect eof

#Another example is a script that automates ftp:

# Set timeout parameter to a proper value.

# For example, the file size is indeed big and the network speed is really one problem,

# you'd better set this parameter a value.

set timeout -1

# Open an ftp session to a remote server, and wait for a username prompt.

spawn ftp $remote_server

expect "username:"

# Send the username, and then wait for a password prompt.

send "$my_user_id\r"

expect "password:"

# Send the password, and then wait for an ftp prompt.

send "$my_password\r"

expect "ftp>"

# Switch to binary mode, and then wait for an ftp prompt.

send "bin\r"

expect "ftp>"

# Turn off prompting.

send "prompt\r"

expect "ftp>"

# Get all the files

send "mget *\r"

expect "ftp>"

# Exit the ftp session, and wait for a special end-of-file character.

send "bye\r"

expect eof

#Another example of automated ssh login in user machine

#timeout is a predefined variable in expect which by default is set to 10 sec

#spawn_id is another default variable in expect.

#It is good practice to close spawn_id handle created by spawn command

set timeout 60

spawn ssh $user@machine

while {1} {

  expect {

    eof                          {break}

    "The authenticity of host"   {send "yes\r"}

    "password:"                  {send "$password\r"}

    "*\]"                        {send "exit\r"}

  }

}

wait

close $spawn_id

#######

expect_out(0,start) Index of the first character of the string that matched the entire expression

expect_out(0,end) Index of the last character of the string that matched the entire expression

expect_out(0,string) String that matched the entire expression

expect_out(1..9,start) Index of the first character of the string that matched the pattern enclosed in the 1st - 9th set of parentheses

expect_out(1..9,end) Index of the last character of the string that matched the pattern enclosed in the 1st - 9th set of parentheses

expect_out(1..9,string) String that matched the pattern enclosed in the 1st - 9th set of parentheses

expect_out(buffer) Entire contents of the buffer when a match was found

expect_out(spawn_id) Spawn id of the process which produced the matching pattern

#Expect

Expect is a Tcl/Tk Extension

• Expect is a tool for automating interactive applications such as telnet,

ftp, passwd, fsck, rlogin, tip, etc. Expect really makes this stuff trivial.

• When to use it – Start an external application and simulate user

interaction

Can start telnet to a router

Can send commands

Can capture router outputs

Expect commands:

Spawn – Spawn a new process (telnet,sh etc…)

• Expect – wait for the specified patterns

• Send – Send specified string to the spawned process

• Close – Terminate the process

Expect reserved variables:

timeout – specify the timeout (in seconds) for an expect command, a default is 10

seconds.

• spawn_id – Unique id assigned to each process created by spawn command.

expect_out(buffer) = all characters up to matched pattern.

• expect_out(0,string) = characters that matched “pattern”.

• expect_out(,string) = characters matching subpattern .

Given the character stream “abcdefgh\n”, the variable

contents would be:

expect "b(.*)e([a-z

expect4.7> echo $expect_out(buffer)

abcdefg

expect4.8> echo $expect_out(0,string)

bcdefg

expect4.9> echo $expect_out(1,string)

cd

expect4.10> echo $expect_out(2,string)