Tuesday, November 15, 2011

AS Regular Expressions


Regular expressions are strings of special characters that can be used to search and find character patterns. Within the scope of BGP in Cisco IOS regular expressions can be used in show commands and AS-Path access-lists to match BGP prefixes based on the information contained in their AS-Path.
In order to understand how to build regular expressions we first need to know what the character definitions are for the regex function of IOS. The below table illustrates the regex characters and their usage. This information is contained in the Cisco IOS documentation under the Appendix of Cisco IOS Terminal Services Configuration Guide, Release 12.2.
+------------------------------------------------------+
| CHAR | USAGE                                         |
+------------------------------------------------------|
|  ^   | Start of string                               |
|------|-----------------------------------------------|
|  $   | End of string                                 |
|------|-----------------------------------------------|
|  []  | Range of characters                           |
|------|-----------------------------------------------|
|  -   | Used to specify range ( i.e. [0-9] )          |
|------|-----------------------------------------------|
|  ( ) | Logical grouping                              |
|------|-----------------------------------------------|
|  .   | Any single character                          |
|------|-----------------------------------------------|
|  *   | Zero or more instances                        |
|------|-----------------------------------------------|
|  +   | One or more instance                          |
|------|-----------------------------------------------|
|  ?   | Zero or one instance                          |
|------|-----------------------------------------------|
|  _   | Comma, open or close brace, open or close     |
|      | parentheses, start or end of string, or space |
+------------------------------------------------------+

Some commonly used regular expressions include:
+-------------+---------------------------+
| Expression  | Meaning                   |
|-------------+---------------------------|
| .*          | Anything                  |
|-------------+---------------------------|
| ^$          | Locally originated routes |
|-------------+---------------------------|
| ^100_       | Learned from AS 100       |
|-------------+---------------------------|
| _100$       | Originated in AS 100      |
|-------------+---------------------------|
| _100_       | Any instance of AS 100    |
|-------------+---------------------------|
| ^[0-9]+$    | Directly connected ASes   |
+-------------+---------------------------+

Let’s break some of the above expressions down step-by-step. The first one “.*” says to match any single character (“.”), and then find zero or more instances of that single character (“*”). This means zero or more instances or any character, which effectively means anything.
The next string “^$” says to match the beginning of the string (“^”), and then immediately match the end of the string (“$”). This means that the string is null. Within the scope of BGP the only time that the AS-Path is null is when you are looking at a route within your own AS that you or one of your iBGP peers has originated. Hence this matches locally originated routes.
The next string “^100_” says to match the beginning of the string (“^”), the literal characters 100, and then a comma, an open or close brace, an open or close, a parentheses, the start or end of the string, or a space (“_”). This means that the string must start with the number 100 followed by any non-alphanumeric character. In the scope of BGP this means that routes which are learned from the AS 100 will be matched, as 100 will be the first AS in the path when AS 100 is sending us routes.
The next string “_100$” is the exact opposite of the previous one. This string says to start with any non-alphanumeric character (“_”), followed by the literal characters 100, followed by the end of the string (“$”). This means that AS 100 is the last AS in the path, or in other words that the prefix in question was originated by AS 100.
The next string “_100_” is the combination of the two previous strings with some extra matches. This string means that the literal characters 100 are set between any two non-alphanumeric characters. The first of these could be the start of the string, which would match routes learned from AS 100, while the second of these could be the end of the string, which would match routes originated in AS 100. Another case could be that the underscores represent spaces, in which the string would match any other AS path information as long as “ 100 ” is included somewhere. This would match any routes which transit AS 100, and therefore “_ASN_” is generally meant to match routes that transit a particular AS as defined by the number “ASN”.
The final string “^[0-9]+$” is a little more complicated match. Immediately we can see that the string starts (“^”), and we can see later that it ends (“$”). In the middle we see a range of numbers 0-9 in brackets, followed by the plus sign. The numbers in brackets mean that any number from zero to nine can be matched, or in other words, any number. Next we have the plus sign which means one or more instances. This string “[0-9]+” therefore means one or more instance of any number, or in other words any number including numbers with multiple characters (i.e. 1, 12, 123, 1234, 12345678, etc.). When we combine these all together this string means routes originated in any directly connected single AS, or in other words, the routes directly originated by the peers of your AS.
Now let’s look at a more complicated match, and using the above character patterns we will see how we can construct the expression step by step. Suppose we have the following topology below, where we are looking at the network from the perspective of AS 100.
+--------+ +--------+ +--------+ +--------+
| AS 200 |-| AS 201 |-| AS 202 |-| AS 203 |\
+--------+ +--------+ +--------+ +--------+ \
                                             \
           +--------+ +--------+ +--------+\  \
           | AS 300 |-| AS 301 |-| AS 302 | \  \
           +--------+ +--------+ +--------+  \  -+--------+
                                              >--| AS 100 |
                      +--------+ +--------+  /  -+--------+
                      | AS 400 |-| AS 401 | /  /
                      +--------+ +--------+/  /
                                             /
                                 +--------+ /
                                 | AS 500 |/
                                 +--------+
AS 100 peers with ASes 203, 302, 401, and 500, who each have peers as diagramed above. AS 100 wants to match routes originated from its directly connected customers (ASes 203, 302, 401, and 500) in addition to routes originated from their directly connected customers (ASes 202, 301, and 400). The easiest way to create this regular expression would be to think about what we are first trying to match, and then write out all possibilities of these matches. In our case these possibilities are:
203
203 202
302
302 301
401
401 400
500
Now we could simply create an expression with multiple lines (7 lines to be exact) that would match all of the possible AS paths, but suppose that AS 100 wants to keep this match as flexible as possible so that it will apply to any other ASes in the future. Now let’s try to generalize the above AS-Path information into a regex.
First off we know that each of the matches is going to start and going to end. This means that the first character we will have is “^” and the last character is “$”. Next we know that between the “^” and “$” there will be either one AS or two ASes. We don’t necessarily know what numbers these ASes will be, so for the time being let’s use the placeholder “X”. Based on this our new possible matches are:
^X$
^X X$
Next let’s reason out what X can represent. Since X is only one single AS, there will be no spaces, commas, parentheses, or any other special type characters. In other words, X must be a number. However, since we don’t know what the exact path is, we must take into account that X may be a number with more than one character (i.e. 10, 123, or 10101). This essentially equates to one or more instance of any number zero through nine. In regular expression syntax our two matches would therefore now read:
^[0-9]+$
^[0-9]+ [0-9]+$
This expressions reads that we either have a number consisting of one or more characters zero through nine, or a number consisting of one or more characters zero through nine followed by a space and then another number consisting of one or more characters zero through nine. This brings our expression down to two lines as opposed to our original seven, but let’s see how we can combine the above two as well. To combine them, first let us compare what is different between them.
^[0-9]+$
^[0-9]+ [0-9]+$
From looking at the expressions it is evident that the sequence “ [0-9]+” is the difference. In the first case “ [0-9]+” does not exist in the expression. In the second case “ [0-9]+” does exist in the expression. In other words, “ [0-9]+” is either true or false. True or false (0 or 1) is represented by the character “?” in regex syntax. Therefore we can reduce our expression to:
^[0-9]+ [0-9]+?$
At this point we run into a problem with the order of operations of the regex. As denoted above the question mark will apply only to the plus sign, and not to the range [0-9]. Instead, we want the question mark to apply to the string “ [0-9]+” as a whole. Therefore this string needs to be grouped together using parentheses. Parentheses are used in regular expressions as simply a logical grouping. Therefore our final expression reduces to:
^[0-9]+( [0-9]+)?$
Note that to match a question mark in IOS, the escape sequence CTRL-V or ESC-Q must be entered first, otherwise the IOS parser will interpret the question mark as an attempt to invoke the context sensitive help.


Tuesday, July 12, 2011

RAID5/6 (using ZFS) in FreeBSD 8.x


Ok, FreeBSD still lacks a decent RAID5 implementation within its core system (some people use the geom_raid5 3rd party module that you can find in freenas) – but with ZFS moved into production status in freebsd 8 now we can use this.
ZFS supports various raid levels.  We will use RAID5 in this example – I’ll explain how to use RAID6 later in the article.
Ok, for my example I will use 6 x 2TB hard drives freshly installed in my system (listed as ad10 ad12 ad14 ad16 ad18 ad20 in dmesg) to generate a RAID5 raid set, giving 5 x 2TB of usable space and capable of a single disk failure without loss of data.  Remember, you need a minimum of 3 disks to do RAID5, and you get N-1 capacity (N-2 for RAID6)
First, we need to load ZFS into the system… add the following into your /boot/loader.conf:
vfs.zfs.prefetch_disable=”1″
zfs_load=”YES”
This will cause ZFS to load in the kernel during each boot.  The prefetch_disable is set by default on servers with less than 4GB of ram, but it’s safe to add it anyway.  I’ve found this produces far more stable results in live systems so go with it ;-)
Next, add the following into your /etc/rc.conf file:
zfs_enable=”YES”
This will re-mount any ZFS filesystems on every boot, and setup any necessary settings on each boot.
Now, we will add all 6 disks into a raid5 set called ‘datastore’ – run the following as root:
zpool create datastore raidz ad10 ad12 ad14 ad16 ad18 ad20
‘raidz’ is ZFS’s name for RAID5 – to do RAID6 you would use ‘raidz2′ instead.  You can confirm the command was successful with zpool status as follows:
pool: datastore
state: ONLINE
scrub: none
config:
NAME        STATE     READ WRITE CKSUM
datastore   ONLINE       0     0     0
raidz1    ONLINE       0     0     0
ad10    ONLINE       0     0     0
ad12    ONLINE       0     0     0
ad14    ONLINE       0     0     0
ad16    ONLINE       0     0     0
ad18    ONLINE       0     0     0
ad20    ONLINE       0     0     0
errors: No known data errors
This shows the raid set is online and healthy.  When there are problems, it will drop to DEGRADED state.  If you have too many disk failures, it will show FAULTED and the entire array is lost (in our example we would need to lose 2 disks to cause this, or 3 in a RAID6 setup)
Now we will set the pool to auto-recover when a disk is replaced, run the following as root:
zpool set autoreplace=on datastore
This will cause the array to auto-readd when you replace a disk in the same physical location (e.g. if ad16 fails and you replace it with a new disk, it will re-add the disk to the pool)
You will now notice that you have a /datastore folder with the entire storage available to it.  you can confirm this with zfs list as follows:
NAME             USED  AVAIL  REFER  MOUNTPOINT
datastore       2.63T  6.26T  29.9K  /datastore
You now have a working RAID5 (or RAID6) software raid setup in FreeBSD.
Generally to setup RAID6 instead of RAID5 you replace the word raidz with raidz2.  RAID5 allows for a single disk failure without data loss, RAID6 allows for a double disk failure without data loss.
After a disk failure, run zpool status to ensure the state is set to ONLINE for all the disks in the array then run the command zpool scrub datastore to make zfs rebuild the array.  Rebuilding takes time (it rebuilds based on used data so the more full your array the longer the rebuild time!) – once it’s completed the scrub or “resilver” process, your array will return back to ONLINE status and be fully protected against disk failures once again.
As this process can take (literally) hours to complete some people prefer a RAID6 setup to allow for a 2nd disk failure during those few hours.  This is a decision you should make based on the importance of the data you will store on the array!

LINK URL: https://www.dan.me.uk/blog/2010/02/07/raid56-using-zfs-in-freebsd-8-x/

Sunday, May 29, 2011

Thinkpad TrackPoint Scrolling in Ubuntu (Maverick) 10.10 - IBM T60


My fellow citizens, our long national nightmare of having to use bad pointing devices is over.
Starting with Ubuntu Maverick/10.10, configuring scrolling with the middle button and the TrackPoint is now really easy:
  • Install gpointing-device-settings (sudo aptitude install gpointing-device-settings)
  • Run gpointing-device-settings &, enable the Use wheel emulation using button 2, and enable both vertical and horizontal scrolling.


(gpointing-device-settings is not new but Maverick is the first time it has really worked well for me.)


Saturday, April 23, 2011