Usage

[root@william ]# bash sslreport.txt
Usage: sslreport.sh [ NETWORK / MASK ] [ NETWORK NAME ]

Requirements

SSLReport uses both the nmap and the sslscan binaries. SSLReport will report any errors should it fail to find these binaries.

[root@william images]# bash sslreport.txt 10.1.1.0/23 home
Checking for Binaries .....
 *Successful execution of /usr/bin/nmap
 *Unsuccessful execution of /usr/bin/sslscan

Download

The script can be download here.

Copy Image

First of all the image is copied from a TFTP server.

Note : The option 'secondary' is used to ensure that the primary image is not overwritten. This provides the ability to boot into the previous (primary image) in the event of failure.

# copy tftp flash [tftp server ip] ASR12301c.bin secondary

Set Boot

Next the configuration is set to boot from the primary image.

adx(config)#boot system flash secondary

Reboot

Next reboot the system.

adx(config)# reload

Verify

Once the device has reloaded run the version command to confirm that the device is running the new version of code.

adx# sh version | i System Version
System Version 12.3.01cT403 Oct 28 2011 16:22:37 PDT label: ASR12301c

When running the command 'show service-policy interface outside set connection detail' the SSH thread incorrectly tries to access the wrong memory area. This results in the Cisco ASA rebooting. 

This issue is also referenced within the Cisco caveat CSCtf22329.

Solution

Upgrade to software version 8.2.5(22).

set applications application UDP-PORT-RANGE protocol udp destination-port 5000-6999

Once created you can then add this to a group. This group can then be added to the necessary policy. Below shows an example:

set groups GRP-PORTS
set applications application-set GRP-PORTS application RDP-3389
set applications application-set GRP-PORTS application UDP-PORT-RANGE

set security policies from-zone untrust to-zone trust policy example-policy match application GRP-PORTS

Types

Sticky - Traffic is sent to the same server based on the clients IP for duration of the sticky timeout duration.
Track Port - A group of ports are defined. One of the ports is defined as a primary port. If a connection is initiated to the the primary port and then any further connections to any of the other ports within the group, persistence is applied. However if a connection is destined to a non-primary port(s) no persistence is applied.
Track Group - A group of ports are defined. If a connection goes to either of these ports then persistence is applied. This is typically used when SSL termination is used and traffic is sent to a different backend port.
Cookie Insert - A cookie is inserted into the HTTP header. The cookie is then used for persistence.
Cookie Passive - A cookie is inserted by the server. The cookie is then used to balance traffic for the remainder of the session to the same server.
Cookie Hashing - A cookie is generated on the server. The ADX generates a hash of this cookie and balances traffic based upon the cookies hash value.

Sticky

Here persistence (on a per client IP basis) is applied to port 80.

server virtual VIP1 192.168.1.3
  port sticky 80

Track Port

The track command is configured using the following construct 'track <primary-port> <port>'. Within this example if traffic is sent to the primary port (80) and any further connections to the other ports within the group (port 23 and 69) then connections are sent to the same server. If any connections are destined for any port that is not defined as the primary port no persistence is applied.

server virtual VIP1 192.168.1.3
  port sticky 80
  port sticky 69
  port sticky 23
  track 80 23 69

Track Group

Within this example traffic destined for either port 80, 69 and 23 is distributed to the same server.

server virtual VIP1 192.168.1.3
  port 80
  port 69
  port 23
  track-group 80 69 23

Cookie Insert

Within this example cookie persistence is achieved by injecting a cookie (containing the server ID). The value of this cookie is then used to maintain persistence to the necessary server.

csw-rule "r1" header "cookie" search "ServerID="

csw-policy "p1"
    match "r1" persist offset 0 length 4 group-or-server-id
    default forward 1
    default rewrite insert-cookie "ServerID"

server real RS1 10.1.1.1
    port http
    port http url "HEAD /"
    port http server-id 1211
    port http group-id 1 1

server real RS2 10.1.1.2
    port http
    port http url "HEAD /"
    port http server-id 1212
    port http group-id 1 1

server virtual VIP1 192.168.1.3
    port http
    port http csw-policy "p1"
    port http csw
    bind http RS1 http RS2 http

Cookie Passive

Within the following example the string "ServerID=" is searched for within the header 'cookie'. The value following this string is then used to balance the traffic.

csw-rule "r1" header "cookie" search "ServerID="

csw-policy "p1"
    match "r1" persist offset 0 length 4 group-or-server-id
    default forward 1

server real RS1 10.1.1.1
    port http
    port http url "HEAD /"
    port http server-id 1211
    port http group-id 1 1

server real RS2 10.1.1.2
    port http
    port http url "HEAD /"
    port http server-id 1212
    port http group-id 1 1

server virtual VIP1 192.168.1.3
    port http
    port http csw-policy "p1"
    port http csw
    bind http RS1 http RS2 http

Cookie Hashing

Within the following example the string "JSessionID=" is searched for within the header 'cookie'. The value following this string is then hashed and used to balance the traffic.

csw-rule r1 header cookie pattern “JSessionID=”

csw-policy p1
    match r1 persist offset 0 length 16 persist-to-hash-bucket
    default forward 1
   
server real RS1 10.1.1.1
    port 80
    port 80 group-id 1 1

server real RS2 10.1.1.2
    port 80
    port 80 group-id 1 1

server virtual VIP1 192.168.1.3
    port http
    port http csw-policy p1
    port http csw
    bind http RS1 80 RS2 80

When sending traffic across a network, computers use something called an MTU (Maximum Transmission Unit). This (network interface) setting dictates the size of the largest frame it can send across the network.

Below shows the MTU default,

Example : A server is wanting to send an Ethernet Packet using TCP. The default MTU would be 1500 which excludes the Ethernet headers and trailers. The TCP header would use 20 bytes, with another 20 bytes used for the IP header. Leaving us with 1460 bytes for the date payload.

What does this have to do with PMTU Discovery ?

When a server sends its traffic across the network (internet), one of the network devices (routers, etc) may have an MTU smaller then the sending computer.

In this scenario 2 things can happen.

• Fragmentation - If the sending computer has not set the DF (Don`t Fragment) bit, then the traffic will be fragmented.
• Path MTU Discovery - If the DF (Don`t Fragment) bit is set the network device will send an ICMP packet back to the sending computer stating its MTU size.

Modern systems tend not to use fragmentation due to the overhead involved in sending multiple packets, not to mention the various security issues involved.

A closer look at PMTU Discovery ?

When the networking node (router) recieves the frame which is larger then the outgoing interfaces MTU it checks for the DF bit. If the DF bit is set, it is unable to fragment the packet so it discards the packet and sends a ICMP Fragmentation Needed message back to the sender.

This message is stating that it needs to fragment the packet but it is unable to because of the DF bit being set. RFC 1911 expands this ICMP message to incorporate the MTU of the interface that is unable to fragment the packet. Once the sender has received this ICMP message it can then adjust its MTU so that it can send a packet at the correct size so that the router is then able to pass it on.

What is a PMTU Black Hole ?

A PMTU black hole is where the ICMP message doesn't reach the sending host to inform it that it needs to adjust its MTU. This can be down to the router not sending the ICMP message or the ICMP message being blocked on the way back to the sender, 

In this scenario the sender is waiting for an acknowledgment for its sent packet. The destination is still waiting for its packet, and the whole session falls down.

Below shows you the 2 scenarios,

ICMP messages not being sent

ICMP messages being blocked

This causes a number of issues. Such as the client finding they might be unable to access one site, this is normally an SSL based site due to the data payload overhead of SSL.

The most common scenario that I see is where a page will load but it will take ages to do so. This can be down to black hole detection. In this case the sending server doesn't receive an acknowledgment for their sent packets so after a certain amount of retries it reduces its MTU (and in turn MSS) and tries resend the packets, in a hope that the reduced packets will make it through to the destination. Of course this greatly increases the delay when trying open the web page, and can cause much confusion to the client and system admins. 

Troubleshooting Steps

Using the ping command we can troubleshoot and hopefully find the hop in which the blackhole exists. By using the ping commands to send a packet at various sizes with the DF bit set we can see if the router sends back the correct ICMP message, what the PMTU is, and where the black hole actually is.

These steps are based on the Windows ping command. We will use the following switches in this troubleshooting exercise,

1. -l Sets the size of the payload with the ICMP packet.
2. -f Sets the DF (Do Not Fragment) bit.

So to calculate a ICMP packet that will have an MTU of 1500 will we use the following,

• MTU = 20 Bytes (IP Header) + 8 bytes (ICMP Header) + 1472 Bytes (ICMP Payload)

So when selecting the Maximum Transmission unit you want to send, minus 28 bytes from your total MTU size to obtain your ICMP payload size.

First we will send a packet with an MTU of 1500,

From the response we can determine the following,

• The Ping is successful and you receive echo replies - Each hop has an MTU of 1500.
• You receive the message "Packet needs to be fragmented but DF set" - The router has successfully sent the correct ICMP response (required for PMTUD)
• You receive the messages "Request timed out" - The router hasn't send the required ICMP message needed for PMTUD.

By increasing and decreasing the ICMP payload size (-l switch) you can determine the Path MTU to your destination.

ping [destination ip/name] -f -l 1472 

References

• http://www.faqs.org/rfcs/rfc2923.html
• http://packetlife.net/blog/2008/aug/18/path-mtu-discovery/
• http://support.microsoft.com/kb/314825
• http://www.netheaven.com/pmtu.html

Define Traffic

First of all we define which traffic the MPF policy will be applied to. In the example below we exclude the host 8.8.8.8 whilst inspecting all other traffic.

cisco-asa(config)# access-list mpf-policy-acl extended deny ip host 8.8.8.8 any
cisco-asa(config)# access-list mpf-policy-acl extended permit ip any any

Create Class-map

Next we assign the previously created access-list to a class-map.

cisco-asa(config)# class-map mpf-policy
cisco-asa(config-cmap)# match access-list mpf-policy-acl
cisco-asa(config-cmap)# exit

Create Policy-map

Then a policy-map is created and the necessary connection limits defined.

cisco-asa(config)# policy-map mpf-policy-map
cisco-asa(config-pmap)# class mpf-policy
cisco-asa(config-pmap-c)# set connection conn-max 9500
cisco-asa(config-pmap-c)# set connection embryonic-conn-max 5000
cisco-asa(config-pmap-c)# set connection per-client-embryonic-max 100
cisco-asa(config-pmap-c)# set connection per-client-max 75
cisco-asa(config-pmap-c)# exit
cisco-asa(config-pmap)# exit

Assign to Interface

Finally the policy map is assigned against a service-policy and interface

cisco-asa(config)# service-policy mpf-policy-map interface outside

To clear the Cisco ASA connection counter the following command is used.

cisco-asa(config)# clear resource usage resource conns

The Cisco CSS is showing a high level of CPU usage, even though the networking throughput does not appear excessively high nor is there a large number of EQL or DQL`s configured.

CSS11501# sh system-resources cpu
Chassis CPU Utilizations
Module Name Module 5Sec 1Min 5Min
----------------------------------------------------
CSS501-SCM-INT 1 90% 88% 75%
CSS501-SSL-C-INT 2 0% 0% 0%

Solution

Though there can be a number of causes to high CPU, within this article we look at the CPU resource consumption being due to the flow table.
Due to a high level of connections transversing the CSS the LoadBalancer uses CPU resource to build the subsequent flows within the flow table.
A typically example being DNS traffic. The following example shows the required commands need to disable flow created for UDP/53.

flow-state 53 udp flow-disable nat-enable

mysqladmin create [new db name] -u [username] --password=[password] && \
mysqldump -u [username] ---password=[password] [old db name] | mysql -u [username] --password=[password] -h [host] [new db name]