+

Search Tips   |   Advanced Search

Parsing semi-structured text with Ansible

The cli_parse module parses semi-structured data such as network configurations into structured data to allow programmatic use of the data from that device. You can pull information from a network device and update a CMDB in one playbook. Use cases include automated troubleshooting, creating dynamic documentation, updating IPAM (IP address management) tools and so on.


Understanding the CLI parser

The ansible.netcommon collection version 1.2.0 or later includes the cli_parse module that can run CLI commands and parse the semi-structured text output. You can use the cli_parse module on a device, host, or platform that only supports a command-line interface and the commands issued return semi-structured text. The cli_parse module can either run a CLI command on a device and return a parsed result or can simply parse any text document. The cli_parse module includes cli_parser plugins to interface with a variety of parsing engines.


Why parse the text?

Parsing semi-structured data such as network configurations into structured data allows programmatic use of the data from that device. Use cases include automated troubleshooting, creating dynamic documentation, updating IPAM (IP address management) tools and so on. You may prefer to do this with Ansible natively to take advantage of native Ansible constructs such as:

By parsing semi-structured text into Ansible native data structures, you can take full advantage of Ansible's network modules and plugins.


When not to parse the text

You should not parse semi-structured text when:


Parsing the CLI

The cli_parse module includes the following cli_parsing plugins:

native

The native parsing engine built into Ansible and requires no addition python libraries

xml

Convert XML to an Ansible native data structure

textfsm

A python module which implements a template based state machine for parsing semi-formatted text

ntc_templates

Predefined textfsm templates packages supporting a variety of platforms and commands

ttp

A library for semi-structured text parsing using templates, with added capabilities to simplify the process

pyats

Uses the parsers included with the Cisco Test Automation & Validation Solution

json

Converts JSON output at the CLI to an Ansible native data structure

Although Ansible contains a number of plugins that can convert XML to Ansible native data structures, the``cli_parse`` module runs the command on devices that return XML and returns the converted data in a single task.

Because cli_parse uses a plugin based architecture, it can use additional parsing engines from any Ansible collection.

The ansible.netcommon.native and ansible.netcommon.json parsing engines are fully supported with a Red Hat Ansible Automation Platform subscription. Red Hat Ansible Automation Platform subscription support is limited to the use of the ntc_templates, pyATS, textfsm, xmltodict, public APIs as documented.


Parsing with the native parsing engine

The native parsing engine is included with the cli_parse module. It uses data captured using regular expressions to populate the parsed data structure. The native parsing engine requires a YAML template file to parse the command output.

Networking example

This example uses the output of a network device command and applies a native template to produce an output in Ansible structured data format.

The show interface command output from the network device looks as follows:

Create the native template to match this output and store it as templates/nxos_show_interface.yaml:

This native parser template is structured as a list of parsers, each containing the following key-value pairs:

The following example task uses cli_parse with the native parser and the example template above to parse the show interface command from a Cisco NXOS device:

Taking a deeper dive into this task:

ansible.netcommon.native parsing engine is fully supported with a Red Hat Ansible Automation Platform subscription.

Lastly in this task, the set_fact option sets the following interfaces fact for the device based on the now-structured data returned from cli_parse:

Linux example

You can also use the native parser to run commands and parse output from Linux hosts.

The output of a sample Linux command (ip addr show) looks as follows:

Create the native template to match this output and store it as templates/fedora_ip_addr_show.yaml:

The shared key in the parser template allows the interface name to be used in subsequent parser entries. The use of examples and free-spacing mode with the regular expressions makes the template easier to read.

The following example task uses cli_parse with the native parser and the example template above to parse the Linux output:

This task assumes you previously gathered facts to determine the ansible_distribution needed to locate the template. Alternately, you could provide the path in the parser/template_path option.

Lastly in this task, the set_fact option sets the following interfaces fact for the host, based on the now-structured data returned from cli_parse:


Parsing JSON

Although Ansible will natively convert serialized JSON to Ansible native data when recognized, you can also use the cli_parse module for this conversion.

Example task:

Taking a deeper dive into this task:

The use of ansible.netcommon.json is fully supported with a Red Hat Ansible Automation Platform subscription


Parsing with ntc_templates

The ntc_templates python library includes pre-defined textfsm templates for parsing a variety of network device commands output.

Example task:

Taking a deeper dive into this task:

Red Hat Ansible Automation Platform subscription support is limited to the use of the ntc_templates public APIs as documented.

This task and and the predefined template sets the following fact as the interfaces fact for the host:


Parsing with pyATS

pyATS is part of the Cisco Test Automation & Validation Solution. It includes many predefined parsers for a number of network platforms and commands. You can use the predefined parsers that are part of the pyATS package with the cli_parse module.

Example task:

Taking a deeper dive into this task:

Red Hat Ansible Automation Platform subscription support is limited to the use of the pyATS public APIs as documented.

This task sets the following fact as the interfaces fact for the host:


Parsing with textfsm

textfsm is a Python module which implements a template-based state machine for parsing semi-formatted text.

The following sample``textfsm`` template is stored as templates/nxos_show_interface.textfsm

The following task uses the example template for textfsm with the cli_parse module.

Taking a deeper dive into this task:

Red Hat Ansible Automation Platform subscription support is limited to the use of the textfsm public APIs as documented.

This task sets the following fact as the interfaces fact for the host:


Parsing with TTP

TTP is a Python library for semi-structured text parsing using templates. TTP uses a jinja-like syntax to limit the need for regular expressions. Users familiar with jinja templating may find the TTP template syntax familiar.

The following is an example TTP template stored as templates/nxos_show_interfaces.ttp:

The following task uses this template to parse the show interface command output:

Taking a deeper dive in this task:

The task sets the follow fact as the interfaces fact for the host:


Converting XML

Although Ansible contains a number of plugins that can convert XML to Ansible native data structures, the``cli_parse`` module runs the command on devices that return XML and returns the converted data in a single task.

This example task runs the show interface command and parses the output as XML:

Red Hat Ansible Automation Platform subscription support is limited to the use of the xmltodict public APIs as documented.

This task sets the interfaces fact for the host based on this returned output:


Advanced use cases

The cli_parse module supports several features to support more complex uses cases.


Provide a full template path

Use the template_path option to override the default template path in the task:


Provide command to parser different than the command run

Use the command suboption for the parser to configure the command the parser expects if it is different from the command cli_parse runs:


Provide a custom OS value

Use the os suboption to the parser to directly set the OS instead of using ansible_network_os or ansible_distribution to generate the template path or with the specified parser engine:


Parse existing text

Use the text option instead of command to parse text collected earlier in the playbook.


See also

Next Previous