Design of the Bridge Keeper

travis+dfd@subspacefield.org

1  Metadata

1.1  Copyright and Distribution Control

The latest version is always here:
http://www.subspacefield.org/security/dfd_keeper/
 
This is a copyrighted work, with some rights reserved. This work is licensed under the Creative Commons Attribution-Noncommercial-No Derivative Works 3.0 United States License (http://creativecommons.org/licenses/by-nc-nd/3.0/us/). This means you may redistribute it for non-commercial purposes, and that you must attribute me properly (without suggesting I endorse your work).

1.2  What is a Dynamic Firewall Daemon?

Read the DFD paper: http://www.subspacefield.org/security/dfd/
The dynamic firewall daemon (DFD) is a concept, whereas dfd_keeper is an implementation of that concept.

1.3  Where to Download

http://www.subspacefield.org/security/dfd_keeper/code

1.4  Terminology

2  Overview

The dynamic firewall daemon manages your firewall rules, providing a cross-platform text-based API so that clients (you, or an automated script) can all connect to it and request changes. Your python script defines what sort of changes are allowed. It is, essentially, firewall middleware, and frees clients from having to know anything about the firewall's configuration.
The basic usage of this is that you learn enough python to write a script which defines three things:
rules
meaning firewall rules, as python strings
variables
get expanded by DFD (and they are not python nor pf.conf variables)
commands
are things that users type, as well as the python code which executes them
This is made easy by importing from the dfd_keeper module. In fact, there already is an example script called keeper_example.py that gives you a functional firewall, in many situations.
Your script decides to listen on a TCP port, and goes into the main loop of execution. Clients may then connect to it and issue text-based commands which you have defined, and those commands may affect the variables you have defined, and those variable changes affect how rules expand. Connecting to it is possible through telnet or better yet, netcat, since it speaks both EOL conventions.
The commands you have defined must conform to a specific pattern, but you don't even have to put them in a dispatch table because dfd_keeper finds them and extracts the help message through introspection.
The most important thing to know about dfd_keeper's implementation is that it is written in python and uses the Twisted module to support asynchronous I/O. This enables it to handle multiple clients at once without worrying about threads or multiple processes. However, this means that instead of coding control-flow constructs, you instead write code to handle events, and the main control loop is nearly invisible to you. One of the consequences of this is that when it does a blocking call, for example when it calls pfctl to load a new rule set, the main loop waits on it. This can be a while if pfctl has to look up DNS name entries, but this is a relatively small price to pay for not having the concerns about thread deadlock, race conditions, mutexes, and semaphores.
The second most important thing to understand is that dfd_keeper is not a program, but rather a library or framework for creating programs that control your firewall. There is an example, keeper_example.py which shows how you do that. Basically, you set some things up, define some commands, and then start the main event loop (called the reactor) which won't exit until you stop the program via its command interface.
Once the main event loop is running, you can connect to DFD by using netcat (nc) or telnet. The server is listening on a port, and you telnet there and issue text-based commands to interact with the firewall.

2.1  Features

2.2  Sample Transcript

http://www.subspacefield.org/security/dfd_keeper/code/sample_transcript.txt

2.3  How Do I Use It?

Install the files  
The distro has an INSTALL file but basically you need to install these packages. Fortunately, they exist in package and ports form on the latest OpenBSD.
Install dfd_keeper  
Download the source (see 1.3), unpack it properly, go in the dfd_keeper directory, and run this as root:
python ./setup.py install
Create your own keeper script  
You can start with the keeper_example.py in the demonstration.
Run it in test mode first  
./keeper_example.py -test
Connect to it from another window  
nc -v -v localhost 8007
Play with it a bit  
Start with "help".
Modify the script to do what you want  
You'll have to be able to understand python, but it's easy to learn
Run it in test and syntax mode to double-check it does what you want  
./keeper_example.py -syntax
Run it from /etc/rc.local  
cp keeper_example.py /etc/keeper.py
if [ -x /etc/keeper.py ]; then /etc/keeper.py > > /var/log/keeper 2>&1 &; fi

3  Gory Details

3.1  Rendering

Basically, rendering means converting the python objects in memory into firewall rules. This is not a simple stringification, because rules may be invalid (not to be rendered), or they may refer to variables which are not defined, or they may need to be removed because they have expired (timed out).
Essentially, rendering starts at the top of the container hierarchy, and each level asks the lower levels to render (if they are valid) and then concatenates the results in a list. Thus, the rule containers start as a tree, but end up with a linear list of rules. At the very bottom level, the rules themselves have substitutions performed on them and end up as strings.
The rule containers (i.e. everything but the bottom of the hierarchy) must inherit apply_mixin and render_mixin, whereas only the bottommost layer inherits from rule_base.

3.2  Variables

During rendering, DFD does some variable substitution; any variable (denoted with a dollar sign prefixed to its name) is expanded to its value within a certain namespace. Generally you define this namespace inside your script, and then set it up for reference prior to entering the main loop. This process is a bit klugey, but it works.
Variables are considered dynamic state and are saved between invocation, whereas rules are defined by the script and are not saved. If the list of variables used by the rules varies from the list used in the prior invocation, the prior values are not restored.

4  Rule Classes

4.1  named_list

This is a structure I created which is a cross between a dictionary and a list. You can think of it as a dictionary which remembers the order you added things to it, or two lists which allow lookup by key in the key list into the corresponding value on the value list. It is internally implemented as two lists.
The importance of this is that I wanted people to be able to add rules to the various sections of the ruleset without having to worry about the order that pf enforces. By creating entries named after the sections of the pf.conf, I was able to let people add to any of them, and yet serialize all the sections in the right order before providing it to pf.

4.2  apply_mixin

When all rule containers inherit this, it makes it possible to do something to every contained rule. This is accomplished as a depth-first traversal, and the method which it finally calls also has a list of ancestors containers, which allows it to do things like remove itself from its parent easily.

4.3  render_mixin

It has a method for finding out if it is valid to participate in rendering. It also has a render method that will render all enclosed rules in this class in a linear order; to accomplish this, it uses a helper function called flatten. Basically, a list of lists will flatten to a single list. Rendering is done recursively and during rendering it flattens at every level, so no matter how many levels deep, the result which bubbles up to the top will be a list of rules (the way this is encoded is a bit weird, and could be simpler). Render also has the important function of expanding all variables it sees with their values from some namespace. This allows you to modify DFD variables and have those changes appear in rules. It uses the traditional way of marking a variable; it prefixes it with a dollar sign.
This also removes sub-items which return a true response to expired method call. It also makes the default to return False.

4.4  rule_container

This basically inherits from apply_mixin and render_mixin, and serves as something other rule containers may derive from.

4.5  ruleset

This is the class for the top-level ruleset used by pf. It inherits from rule_container (of course) and has seven rule_holders named after the seven parts of a pf.conf.
It is an enumerated list, with sections familiar to any pf user:

4.6  rule_holder

A generic container for holding rules, inherits from rule_container.
This is derived from a list but it defines a method called make_rule which allows you to create a rule inside of this container. It also defines a __str__ which will convert all the contained rules into a string representation.

4.7  rule

This corresponds to one particular rule. It takes a string as input, and if you call str() on it, it will return that string. It also defines __repr__ for saving state, and a render method which will return the variable-substituted rule according to some namespace that is passed in, assuming it is valid. If it is not valid, or if the variable expansion used an empty value, it will return an empty list instead of a rule. It inherits from rule_base (see )
It allows for rule expiration semantics, so that rules can "die" at a certain time. This can be done at an absolute time in seconds since the Epoch ("expires"), or as a number of seconds from now ("lifetime"). The lifetime keyword is is so that client scripts need not know the current time in order to create a rule that expires a fixed time in the future.
It also lets the runtime know how long until it expires, so the runtime event loop can schedule a pass to expire it. It also has a method called apply that allows you to call a function on a rule and all of its ancestors. It also defines a method called valid that tells us if the rule is active and has not expired. You see, rules can be marked inactive, and they will not render until they become active. This is usually done when you want to turn on or off a group of rules in a programmatic way.

4.8  template

This was inspired by string.Template but it lets us expand rules according to some namespace via the substitute.

4.9  rule_queue

This is a variant of rule_series that will only hold a certain number of rules. Once you get the maximum number, adding a single rule will get rid of the oldest rule in a first-in first-out policy.

5  Run-Time Support Classes

5.1  syslogger

logs to syslog

5.2  outlogger

logs to stdout

6  PF Classes

6.1  ParameterError

Raised when you didn't specify a parameter properly

6.2  FlushError

Raised when pf rejects a state flush

6.3  pf

This is the largest class in the system and mainly deals with interacting with pf.
This python object represents the pf firewall. Generally, you instantiate it, and then tell it to synchronize, and it will then provide all the firewall rules to pfctl via stdin.
When you initialize the pf object (there is no need for more than one), you may pass it a namespace (dictionary) in which to look up DFD variables.
There is a chance to control three logging variables.
You may also put it in one of three modes.
daemon
means it will daemonize itself and then behave as in normal mode
normal
means it will affect the pf rules on this system
syntax
means it will not affect the pf rules, but instead will use pfctl -n to syntax-check your rules
test
means that it isn't going to talk to pf at all; this is handy for testing things out on machines that don't have the pf firewall.

6.4  macro

This represents a pf.conf macro, and can be expanded programmatically.

6.5  table

This represents a pf.conf table, and its string representation puts angle brackets around its name. When defined, it gets a unique name based on a class-level counter.

6.6  setlist

This is a helper class where you can add something to this list, but not if it's already there. Basically this is a cross between a set and a list.

6.7  pflist

This derives from setlist and renders to the "{ blah1 blah2}" syntax that pf expects from lists.

6.8  anchor

This represents an anchor object in the pf.conf syntax. You can tell it what type it is, such as rdr, nat, or binat.

6.9  magic

This setlist represents something which can expand to a simple string, or a pf.conf list of items, depending on how many there are. It was needed back when enclosing a single value in braces wasn't legal pf.conf syntax, and that came in handy when you wanted an object that could hold one or more than one value in the rule. To put it another way, it adds the curly braces around itself if it has more than one object. So this basically is magic because it behaves as either macro (for one value) or pflist (for two values). This was very important when the original pf.conf syntax wouldn't accept braces around a single object.

7  Twisted Classes

7.1  tcp_factory

This factory from the Twisted framework creates pairs of objects, takers and commanders, which represent the connection from a user and the commands that user may enter. These objects have to be linked together to get stuff done.
This class is tailored for tcp command takers and allows you to easily specify a commander and protocol to instantiate. Optionally you may specify a banner string, responses for valid and invalid commands, and the prompt you wish to display. If you wish to disable any of these just make them an empty string. You may also configure the name of the quit command (or None if you don't wish to have one).

7.2  pfctl_process

This is a module that allows the Twisted engine to interact with the pfctl command.

7.3  line_receiver

This class takes care of processing lines of input to or from clients. It assumes all incoming strings end with LF or CR-LF. It strips the carriage return so that people may use telnet, which has that end-of-line convention. It also adds carriage returns in the right place on outbound traffic so that telnet accepts it. Fortunately netcat will ignore them. You have to derive from this and implement line_received to do something with it, since this is an abstract base class.

7.4  stateful_queue

This class appears to allow you to enqueue a number of method calls using append or extend method calls. Each of these triggers the queue to start processing the commands, as long as it is not paused. There are pause and resume commands which handle that.

7.5  asynchronous_command

This is returned by a command to indicate that it will send back output asynchronously, so go ahead and return to the client and await notification.

7.6  tcp_command_taker

This Twisted framework object receives method calls when connections are made, have sent a line of text, It is also responsible for making sure that the user-entered command is not zero-length, and that the exists and has the proper number of arguments. It also handles a few commands inline, such as quitting a connection or stopping the "reactor" altogether. It makes use (by composition) of the stateful_queue class described above. Every time a line comes in, it is broken up by semicolons, and then extended onto the stateful_queue for processing. When a command throws an asynchronous_command exception, it prints the output it has so far and pauses the queue for the async event to restart it

7.7  chronos

When you construct this class and pass it an instance of the pf class, it expires any already-expired rules and schedules callbacks to delete any rules due to expire in the future.

7.8  ipv4.address

This represents a dotted quad

7.9  ipv4.range

This represents a range of addresses using an address and mask (CIDR style)

7.10  sync_proxy

This is a proxy between two pieces of code; here it is being used between pf and pfctl_process. I wrote this for sites which might have a great deal of things going on, such that calling pfctl that often would make it a bottleneck. The sync_proxy can count the number of sync requests, and only send one every sync_every times, where sync_every is given to the constructor and defaults to one.

7.11  helper_functions

This class exists to implement things to ease the difficulty of writing scripts. For example, it handles taking the user's input and dispatching to a command that you have written.

8  Other

There are a few more methods related to daemonizing, handling interrupts and saving data (variables, actually) persistently.

9  TODO



File translated from TEX by TTH, version 3.67.
On 23 Oct 2008, 21:33.