A Hands-on Guide to Multi-Tiered Firewall Changes with Ansible and Batfish (Part 3)

Introduction

Welcome back to part 3. We will now look at an example workflow using the steps and Playbooks previously covered in parts 1 and 2.

WorkFlow

Our workflow will consist of the following steps:

Figure 1 - WorkFlow overview.

With regards to the Dev environment. This is our local Batfish/Ansible environment.

Walk-Through

Let’s now go through our example workflow using the steps and playbooks that we previously saw in part 2.

Create/Validate Snapshot

First of all we will create and validate our Batfish snapshot.

Create

Validate

As you can see from the output, the only issues seen are around AAA config. This won't cause an issue with ACL validation so we can move forward.

Make Changes (Dev)

Next we will make our changes locally (i.e within Dev). Our changes will be to add the new NTP/DNS server IP of 8.8.4.4.

Validate Changes (Dev)

We will now validate our changes. First we will add the IP to our flows defined within playbooks/bf_validate_acls_flows.yml. Once done we will run the playbook to validate that all of the expected flows are permitted through the firewalls, and that no lines are unreachable.

Ok, so our output has flagged 2 issues, one from our flow assertions, and one from our unreachable assertions.

So the first error shows there is an issue with traffic from the DB to 8.8.4.4. We can see via the message RECEIVED(db), DENIED(acl-db (POST_TRANSFORMATION_INGRESS_FILTER))))]), that the ACL acl-db on FW2 is blocking the traffic.

{
    "details": "Found a flow that failed, when expected to succeed\n[{'Flow': Flow(dscp=0, dstIp='8.8.4.4', dstPort=53, ecn=0, fragmentOffset=0, icmpCode=None, icmpVar=None, ingressInterface='db', ingressNode='fw2', ingressVrf=None, ipProtocol='UDP', packetLength=0, srcIp='10.0.2.1', srcPort=0, state='NEW', tag='BASE', tcpFlagsAck=0, tcpFlagsCwr=0, tcpFlagsEce=0, tcpFlagsFin=0, tcpFlagsPsh=0, tcpFlagsRst=0, tcpFlagsSyn=0, tcpFlagsUrg=0), 'Traces': ListWrapper([((RECEIVED(db), DENIED(acl-db (POST_TRANSFORMATION_INGRESS_FILTER))))]), 'TraceCount': 1}]",
    "name": "Confirm that db can reach public DNS/NTP server on FW2:GigEth0/1",
    "status": "Fail",
    "type": "assert_all_flows_succeed"
}

What else do we see? Well we also have an ACL unreachable assertion failure.

{
    "details": "Found unreachable filter line(s), when none were expected\n[{'Sources': ListWrapper(['fw2: acl-db']), 'Unreachable_Line': 'permit udp host 10.0.2.1 host 8.8.4.4 eq ntp', 'Unreachable_Line_Action': 'PERMIT', 'Blocking_Lines': ListWrapper(['deny ip any4 any4']), 'Different_Action': True, 'Reason': 'BLOCKING_LINES', 'Additional_Info': None}, {'Sources': ListWrapper(['fw2: acl-db']), 'Unreachable_Line': 'permit udp host 10.0.2.1 host 8.8.4.4 eq domain', 'Unreachable_Line_Action': 'PERMIT', 'Blocking_Lines': ListWrapper(['deny ip any4 any4']), 'Different_Action': True, 'Reason': 'BLOCKING_LINES', 'Additional_Info': None}]",
    "name": "Confirm that there are no unreachable lines",
    "status": "Fail",
    "type": "assert_filter_has_no_unreachable_lines"
}

Within this message you can see it tells us about our unreachable line and the blocking line. I think you can see the issue here. If we take a look at our ACL, we can clearly see the issue.

access-list acl-db extended permit udp host 10.0.2.1 host 8.8.8.8 eq ntp
access-list acl-db extended permit udp host 10.0.2.1 host 8.8.8.8 eq domain
access-list acl-db extended deny ip any4 any4
access-list acl-db extended permit udp host 10.0.2.1 host 8.8.4.4 eq ntp
access-list acl-db extended permit udp host 10.0.2.1 host 8.8.4.4 eq domain

Let’s fix the issue and rerun our validations.

Our previous issue appears to be resolved, however there is a new issue.

{
    "details": "Found a flow that failed, when expected to succeed\n[{'Flow': Flow(dscp=0, dstIp='8.8.4.4', dstPort=53, ecn=0, fragmentOffset=0, icmpCode=None, icmpVar=None, ingressInterface='db', ingressNode='fw2', ingressVrf=None, ipProtocol='UDP', packetLength=0, srcIp='10.0.2.1', srcPort=0, state='NEW', tag='BASE', tcpFlagsAck=0, tcpFlagsCwr=0, tcpFlagsEce=0, tcpFlagsFin=0, tcpFlagsPsh=0, tcpFlagsRst=0, tcpFlagsSyn=0, tcpFlagsUrg=0), 'Traces': ListWrapper([((RECEIVED(db), PERMITTED(acl-db (POST_TRANSFORMATION_INGRESS_FILTER)), FORWARDED(Routes: static [Network: 0.0.0.0/0, Next Hop IP:10.0.0.13]), PERMITTED(~COMBINED_OUTGOING_ACL~outside~ (PRE_TRANSFORMATION_EGRESS_FILTER)), TRANSMITTED(outside)), (RECEIVED(inside), DENIED(acl-inside (POST_TRANSFORMATION_INGRESS_FILTER))))]), 'TraceCount': 1}]",
     "name": "Confirm that db can reach public DNS/NTP server on FW2:GigEth0/1",
     "status": "Fail",
     "type": "assert_all_flows_succeed"
}

We can see the problematic traffic is our new 8.8.4.4 IP going to UDP/53 and that the traffic makes it to FW2 and then the acl acl-inside denies the traffic. If we look at this acl we see the following:

access-list acl-inside extended permit udp host 10.0.2.1 host 8.8.8.8 eq ntp
access-list acl-inside extended permit udp host 10.0.2.1 host 8.8.8.8 eq domain
access-list acl-inside extended deny ip any4 any4

As you can see this traffic hasn't been permitted. Let’s fix and rerun our validation.

As you can see all of our tests pass successfully. Furthermore Prod has not been touched and all changes/validations have been performed locally within Dev. Niiice.

Push Changes (Prod)

We now push the configuration that is contained within our snapshot out to our production environment. Like so:

Note: The playbook used clears the ACLs before reapplying the configuration. This is because there is no logic within the asa_config around line numbering, so reapplying the same ACLs in different positions for example will not result in the expected ACL configuration. In a production environment though, this caveat would be negated if you have a HA pair by applying the changes to the standby first, performing a failover, and then issuing a HA sync.

Validate Changes (Prod)

To validate our changes in prod we will create a new snapshot and then rerun our validation tests via playbooks/bf_validate_acls_flows.yml.

As you can see the validation passed successfully. GREAT! We have applied our ACL changes to Prod without any issues (or additional CR’s being required!), along with all testing being isolated solely to our Dev environment.

Outro

That concludes this 3-part Hands-on Guide to Multi-Tiered Firewall Changes with Ansible and Batfish. As I'm sure you can see, Batfish brings many benefits when dealing with multi-tier firewall topologies. Although we have covered a lot in this guide, we have only scratched the surface.