Module pexpect#
Module pexpect allows to automate interactive connections such as:
telnet
ssh
ftp
Note
Pexpect is an implementation of expect in Python.
First, pexpect module needs to be installed:
pip install pexpect
The logic of pexpect is:
some program is running
pexpect expects a certain output (prompt, password request, etc.)
after receiving the output, it sends commands/data
last two actions are repeated as many times as necessary
At the same time, pexpect does not implement utilities but uses ready-made ones.
pexpect.spawn#
Class spawn allows you to interact with called program by sending data and
waiting for a response.
For example, you can initiate SSH connecton:
In [5]: ssh = pexpect.spawn('ssh cisco@192.168.100.1')
After executing this line, connection is established. Now you must specify which line to expect. In this case, wait for password request:
In [6]: ssh.expect('[Pp]assword')
Out[6]: 0
Note how line that pexpect expects is written as [Pp]assword. This is a
regex that describes a password or Password string. That is, expect
method can be used to pass a regex as an argument.
Method expect returned number 0 as a result of the work. This number
indicates that a match has been found and that this element with index zero.
Index appears here because you can pass a list of strings. For example,
you can pass a list with two elements:
In [7]: ssh = pexpect.spawn('ssh cisco@192.168.100.1')
In [8]: ssh.expect(['password', 'Password'])
Out[8]: 1
Note that it now returns 1. This means that Password word matched.
Now you can send password using sendline method:
In [9]: ssh.sendline('cisco')
Out[9]: 6
Method sendline sends a string, automatically adds a new line character
to it based on the value of os.linesep and then returns a number
indicating how many bytes were written.
Note
Pexpect has several methods for sending commands, not just sendline.
To get into enable mode expect-sendline cycle repeats:
In [10]: ssh.expect('[>#]')
Out[10]: 0
In [11]: ssh.sendline('enable')
Out[11]: 7
In [12]: ssh.expect('[Pp]assword')
Out[12]: 0
In [13]: ssh.sendline('cisco')
Out[13]: 6
In [14]: ssh.expect('[>#]')
Out[14]: 0
Now we can send a command:
In [15]: ssh.sendline('sh ip int br')
Out[15]: 13
After sending the command, pexpect must be told until what point to read the output.
We specify that it should read untill #:
In [16]: ssh.expect('#')
Out[16]: 0
Command output is in before attribute:
In [17]: ssh.before
Out[17]: b'sh ip int br\r\nInterface IP-Address OK? Method Status Protocol\r\nEthernet0/0 192.168.100.1 YES NVRAM up up \r\nEthernet0/1 192.168.200.1 YES NVRAM up up \r\nEthernet0/2 19.1.1.1 YES NVRAM up up \r\nEthernet0/3 192.168.230.1 YES NVRAM up up \r\nEthernet0/3.100 10.100.0.1 YES NVRAM up up \r\nEthernet0/3.200 10.200.0.1 YES NVRAM up up \r\nEthernet0/3.300 10.30.0.1 YES NVRAM up up \r\nR1'
Since the result is displayed as a sequence of bytes you should convert it to a string:
In [18]: show_output = ssh.before.decode('utf-8')
In [19]: print(show_output)
sh ip int br
Interface IP-Address OK? Method Status Protocol
Ethernet0/0 192.168.100.1 YES NVRAM up up
Ethernet0/1 192.168.200.1 YES NVRAM up up
Ethernet0/2 19.1.1.1 YES NVRAM up up
Ethernet0/3 192.168.230.1 YES NVRAM up up
Ethernet0/3.100 10.100.0.1 YES NVRAM up up
Ethernet0/3.200 10.200.0.1 YES NVRAM up up
Ethernet0/3.300 10.30.0.1 YES NVRAM up up
R1
Session ends with a close call:
In [20]: ssh.close()
Special characters in shell#
Pexpect does not interpret special shell characters such as >,
|, *.
For example, in order make command ls -ls | grep SUMMARY work, shell
must be run as follows:
In [1]: import pexpect
In [2]: p = pexpect.spawn('/bin/bash -c "ls -ls | grep pexpect"')
In [3]: p.expect(pexpect.EOF)
Out[3]: 0
In [4]: print(p.before)
b'4 -rw-r--r-- 1 vagrant vagrant 3203 Jul 14 07:15 1_pexpect.py\r\n'
In [5]: print(p.before.decode('utf-8'))
4 -rw-r--r-- 1 vagrant vagrant 3203 Jul 14 07:15 1_pexpect.py
pexpect.EOF#
In the previous example we met pexpect.EOF.
Note
EOF — end of file
This is a special value that allows you to react to the end of a command or session that has been run in spawn.
When calling ls -ls command, pexpect does not receive an interactive
session. Command is simply executed and that ends its work.
Therefore, if you run this command and set prompt in expect, there is an error:
In [5]: p = pexpect.spawn('/bin/bash -c "ls -ls | grep SUMMARY"')
In [6]: p.expect('nattaur')
---------------------------------------------------------------------------
EOF Traceback (most recent call last)
<ipython-input-9-9c71777698c2> in <module>()
----> 1 p.expect('nattaur')
...
If EOF passed to expect, there will be no error.
Method pexpect.expect#
In pexpect.expect as a value can be used:
regex
EOF - this template allows you to react to EOF exception
TIMEOUT - timeout exception (default timeout = 30 seconds)
compiled regex
Another very useful feature of pexpect.expect is that you can pass
not a single value, but a list.
For example:
In [7]: p = pexpect.spawn('/bin/bash -c "ls -ls | grep netmiko"')
In [8]: p.expect(['py3_convert', pexpect.TIMEOUT, pexpect.EOF])
Out[8]: 2
Here are some important points:
when pexpect.expect is called with a list, you can specify different expected strings
apart strings, exceptions also can be specified
pexpect.expect returns number of element that matched
in this case number 2 because EOF exception is number two in the list
with this format you can make branches in the program depending on the element which had a match
Example of pexpect use#
Example of using pexpect when connecting to equipment and passing show command (file 1_pexpect.py):
import pexpect
import re
from pprint import pprint
def send_show_command(ip, username, password, enable, commands, prompt="#"):
with pexpect.spawn(f"ssh {username}@{ip}", timeout=10, encoding="utf-8") as ssh:
ssh.expect("[Pp]assword")
ssh.sendline(password)
enable_status = ssh.expect([">", "#"])
if enable_status == 0:
ssh.sendline("enable")
ssh.expect("[Pp]assword")
ssh.sendline(enable)
ssh.expect(prompt)
ssh.sendline("terminal length 0")
ssh.expect(prompt)
result = {}
for command in commands:
ssh.sendline(command)
match = ssh.expect([prompt, pexpect.TIMEOUT, pexpect.EOF])
if match == 1:
print(
f"Symbol {prompt} is not found in output. Resulting output is written to
dictionary")
if match == 2:
print("Connection was terminated by server")
return result
else:
output = ssh.before
result[command] = output.replace("\r\n", "\n")
return result
if __name__ == "__main__":
devices = ["192.168.100.1", "192.168.100.2", "192.168.100.3"]
commands = ["sh clock", "sh int desc"]
for ip in devices:
result = send_show_command(ip, "cisco", "cisco", "cisco", commands)
pprint(result, width=120)
This part of function is responsible for switching to enable mode:
enable_status = ssh.expect([">", "#"])
if enable_status == 0:
ssh.sendline("enable")
ssh.expect("[Pp]assword")
ssh.sendline(enable)
ssh.expect(prompt)
If ssh.expect([">", "#"]) does not return index 0, it means that connection
was not switched to enable mode automaticaly and it should be done separately.
If index 1 is returned, then we are already in enable mode, for example,
because device is configured with privilege 15.
Another interesting point about this function:
for command in commands:
ssh.sendline(command)
match = ssh.expect([prompt, pexpect.TIMEOUT, pexpect.EOF])
if match == 1:
print(
f"Symbol {prompt} is not found in output. Resulting output is written to dictionary"
)
if match == 2:
print("Connection was terminated by server")
return result
else:
output = ssh.before
result[command] = output.replace("\r\n", "\n")
return result
Here commands are sent in turn and expect waits for three options:
prompt, timeout or EOF.
If expect method didn’t catch #, value 1 will be returned and in this
case a message is displayed, that symbol was not found. But in both cases,
when a match is found or timeout the resulting output is written to dictionary.
Thus, you can see what was received from device, even if prompt is not found.
Output after script execution:
{'sh clock': 'sh clock\n*13:13:47.525 UTC Sun Jul 19 2020\n',
'sh int desc': 'sh int desc\n'
'Interface Status Protocol Description\n'
'Et0/0 up up \n'
'Et0/1 up up \n'
'Et0/2 up up \n'
'Et0/3 up up \n'
'Lo22 up up \n'
'Lo33 up up \n'
'Lo45 up up \n'
'Lo55 up up \n'}
{'sh clock': 'sh clock\n*13:13:50.450 UTC Sun Jul 19 2020\n',
'sh int desc': 'sh int desc\n'
'Interface Status Protocol Description\n'
'Et0/0 up up \n'
'Et0/1 up up \n'
'Et0/2 admin down down \n'
'Et0/3 admin down down \n'
'Lo0 up up \n'
'Lo9 up up \n'
'Lo19 up up \n'
'Lo33 up up \n'
'Lo100 up up \n'}
{'sh clock': 'sh clock\n*13:13:53.360 UTC Sun Jul 19 2020\n',
'sh int desc': 'sh int desc\n'
'Interface Status Protocol Description\n'
'Et0/0 up up \n'
'Et0/1 up up \n'
'Et0/2 admin down down \n'
'Et0/3 admin down down \n'
'Lo33 up up \n'}
Working with pexpect without disabling commands pagination#
Sometimes the output of a command is very large and cannot be read completely or device is not makes it possible to disable pagination. In this case, a slightly different approach is needed.
Note
The same task will be repeated for other modules in this section.
Example of using pexpect to work with paginated output of show command (1_pexpect_more.py file):
import pexpect
import re
from pprint import pprint
def send_show_command(ip, username, password, enable, command, prompt="#"):
with pexpect.spawn(f"ssh {username}@{ip}", timeout=10, encoding="utf-8") as ssh:
ssh.expect("[Pp]assword")
ssh.sendline(password)
enable_status = ssh.expect([">", "#"])
if enable_status == 0:
ssh.sendline("enable")
ssh.expect("[Pp]assword")
ssh.sendline(enable)
ssh.expect(prompt)
ssh.sendline(command)
output = ""
while True:
match = ssh.expect([prompt, "--More--", pexpect.TIMEOUT])
page = ssh.before.replace("\r\n", "\n")
page = re.sub(" +\x08+ +\x08+", "\n", page)
output += page
if match == 0:
break
elif match == 1:
ssh.send(" ")
else:
print("Error: timeout")
break
output = re.sub("\n +\n", "\n", output)
return output
if __name__ == "__main__":
devices = ["192.168.100.1", "192.168.100.2", "192.168.100.3"]
for ip in devices:
result = send_show_command(ip, "cisco", "cisco", "cisco", "sh run")
with open(f"{ip}_result.txt", "w") as f:
f.write(result)
Now after sending the command, expect method waits for another
option --More-- - sign, that there will be one more page further.
Since it’s not known in advance how many pages will be in the output,
reading is performed in a loop while True. Loop is interrupted if
prompt is met # or no prompt appears within 10 seconds or --More--.
If --More-- is met, pages are not over yet and you have to scroll through
the next one. In Cisco, you need to press space bar to do this (without new line).
Therefore, send method is used here, not sendline - sendline automatically
adds a new line character.
This string page = re.sub(" +\x08+ +\x08+", "\n", page) removes backspace
symbols which are around --More-- so they don’t end up in the final output.