Pipe and Filter Architecture

For this week’s blog I chose an article on the pipe and filter architecture appropriately titled “Pipe-And-Filter.” I chose this article after googling what some of the most common software architectures are and learning that pipe and filter was commonly implemented. This article seemed like a good length with straightforward information and diagrams to help with understanding the material so that is what I chose it.

To begin, this architecture consists of any number of components referred to as filters due to the fact that they filter data before passing it through connectors called pipes to other components. All of the filters work at the same time and this is usually implemented in simpler sequences although is not limited to that.

pipe1

Above is a simple diagram to show how the architecture flows. It’s important to know that filters can transform the input data from any number of pipes. The pipes pass data between filters however it is unidirectional implemented by a buffer until the downstream filter can process it. The pump is where the data originates such as a text file or I/O device. Lastly the sink is the end target of the transformed data such as a file, database, or output to a screen.

One good example of this architecture would be a Unix program. One program’s output can piped into another program’s input.

pipe2

Above is a more complex diagram to show how pipe and filter can start to become complex. Different sources or pumps can interconnect data into their respective streams. An application that uses this architecture will typically link all the components together and then spawn a thread for each filter to run in.

One interesting functionality of this pattern is a recursive filter technique. This is implemented by having a filter inside of another filter.

One common issue with this type of architecture concerns what kind of data types are allowed in a certain pipe. If only one type is allowed, filters need to parse for this which can slow an application down. You may also limit yourself to what pipes can connect to which filters.

After reading this article I have a good idea of pipe and filters main concepts. One thing I wished the article had discussed more in detail would be specific implementations of this architecture. I can’t directly see how I would need to use these concepts in any of the coding I’ve done so far. I can see a general use for this model for an application that takes in a lot of raw data and needs to output it in a useful format for making business decisions. In summary this was a well written article but I need to do some further reading on implementation examples.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s