Tuesday, April 18, 2017

Complex Event Processing for the Internet of Things

Ubiquitous connected sensing devices permeate our world.  These devices capture large amounts of data.  One challenge associated with this data is converting it to actionable insight.  This paper outlines the history of the Internet of Things and explores the application of rule-based complex event processing.

Internet of Things

In 1989, Tim Berners-Lee proposed what would become the World Wide Web (Berners-Lee, 1989).  He proposed organizing data as a web to help CERN, his employer, keep track of documents.  At the time, keyword searching required the manual registration of the keywords associated with a document.  A year later, in 1990, John Romkey demonstrated control of a Sunbeam Deluxe Automatic Radiant Control toaster over the Internet at the Interop conference (Malamud, 1992). The toaster was controlled using Simple Network Management protocol (SNMP).
Mark Weiser’s article on ubiquitous computing was published by Scientific American in 1991 (Weiser, 1991).  The article predicts that computers will fade into the background as they become part of the natural human environment.  The predictions in the article are more striking when we recall that at the time of the writing Microsoft Windows 3.0 had recently been released ("Microsoft Windows 3.0," n.d.).  The Trojan Room Coffee Pot images were streamed using the HTML image tag in 1993 (Gordon & Johnson, n.d.).   When the group connected the page to the Internet, it became the first webcam.
In 1997, Paul Saffo published that sensors would become the next key enabling technology (Saffo, 1997).  He argued that in the 1980s inexpensive microprocessor drove innovation.  Expensive lasers followed microprocessors, benefiting communications and storage.  He foresaw that sensors would be the next enabling technology.  In 1999, Kevin Ashton termed “Internet of Things” during a presentation to Proctor and Gamble (Ashton, 2009).  Ashton was attempting to explain radio-frequency identification (RFID) to a group of executivesA.
The Internet of Things (IoT) is an umbrella term used to describe connection of physical or virtual devices to the Internet (Bassi et al., 2013; Hui, Jiafu, Caifeng, & Jianqi, 2012; Singh & Singh, 2015; Tiburski, Albernaz Amaral, De Matos, & Hessel, 2015).  IoT solutions usually contain three core layers: sensing, network, and application (Cavalcante, Alves, Batista, Delicato, & Pires, 2015).  The sensing, or device, layer is concerned with capturing information.  The network layer is concerned with enabling communication between the sensing layer and the application layer.  Communication through this layer is often bidirectional.  The application layer is concerned with aggregation, event processing, or external communication.  One form of event processing related to the IoT is Complex Event Processing.

Complex Event Processing Background

The term Complex Event Processing (CEP) was coined by David Luckham in 1995 (Leavitt, 2009).  Luckham argued that CEP allowed for the creation of an abstraction hierarchy (D. C. Luckham & Frasca, 1998).   An abstraction hierarchy organizes a system’s activities and operations into a set of layers.  A popular application of this concept is the Open Systems Interconnection model.  Luckham was part of a project that created RAPIDE. 
RAPIDE contained a casual event pattern language (D. Luckham, 2002).  The language allowed for the control of processing flow.  It also specified operations such as filtering and aggregating.  The output of one set of operations could be directed to another.  This allowed for the creation of multiple layers of abstraction.  This was not the first time this sort of analysis was performed on streams of data, but previously that analysis required specialized programs or hand-coded solutions (Leavitt, 2009).
CEP is a subset of Event Stream Processing (ESP) (Weber, Lowe, Malunjkar, & Quinn, 2010).  CEP extended the single event stream processing concept of ESP to include multiple events with a temporal relationship.  Prior to CEP and ESP, real-time information was inserted into a relational database management system.  An inference engine evaluated the data within the database and detected the conditions to trigger an action.  The key distinction is that CEP and ESP do not require the persistence of the real-time information. 
Luckham’s initial implementation of CEP using RAPIDE represented the processing to occur as a set of nodes and arcs.  Because of the popularity of Structured Query Language (SQL) and the vendor experience with relational databases many CEP vendors selected SQL as the means of representing processing rules (Leavitt, 2009).  They retained the basic syntax while extending the language with required constructs. 
In recent systems, the Complex Event Processing pipeline is comprised of three phases (Mehdiyev, Krumeich, Enke, Werth, & Loos, 2015).  Filtering removes items that are not relevant from the processing flow.  Next, matching is attempted using the input from the previous step and the predefined rules.  The last step uses the output from the matching step to attempts to derive events that are more complex.
CEP is stateless, all processing is performed in-memory, and has been applied to machine-to-machine (M2M) communications (Bruns, Dunkel, Masbruch, & Stipkovic, 2015).  CEP is rule-based; this enables a high degree of flexibility when applied to M2M solutions.  M2M is a foundation technology for the IoT.

Application of CEP to IoT

Data produced by the Internet of Things is one of the fastest growing types of data (Barnaghi, Sheth, & Henson, 2013).  The devices that make up the IoT send data to the internet in a stream or in reaction to an event.  Because the devices producing and transmitting this continuous spatiotemporal data are low cost and operate in dynamic environments a continuous flow is not guaranteed.  The transmitted data may be aggregated and summarized, or might be at a low level.  The data the devices produce vary considerably in structure and complexity.

Current Trends

CEP continues to be a relevant technology.  The market is expected to be worth $4.95 billion by 2020 (MarketAndMarkets, 2016).  Microsoft released its version of CEP, named Azure Stream Analytics, in 2014.  Azure Stream Analytics uses an SQL-like syntax to represent the CEP rules.  TIBCO acquired StreamBase on June 11, 2013.  StreamBase is a CEP system that uses a language named StreamSQL EventFlow (StreamBase, 2016).  Oracle offers the Stream Explorer platform (Oracle, 2016).  WSO2 offers Complex Event Processor that is available on-premise or as part of WSO2’s Amazon hosted private cloud.  It uses an SQL-like language to define the event processing (WSO2, 2016).  This is by no means an exhaustive list of offerings.  It appears that most vendors offer a CEP solution. 
The IoT is not a clearly defined concept.  The underlying concepts of ubiquitous computing, inexpensive sensors, highly connected devices, and Big Data Analytics continue to grow and expand.  It is projected that the IoT market will reach $1.7 trillion by 2020 (IDC, 2015).
CEP and IoT are complementary technologies.  IoT produces a stream of data and CEP processes a stream of data.

Research Opportunities

CEP requires that rules developed by an expert in the field.  These rules are typically represented in an SQL-like language.  A subject matter expert usually creates the rules.  There have been several research efforts to auto-discover the rules (Lin, Wu, Huang, & Tseng, 2015; Mehdiyev et al., 2015).  These efforts are useful, but it might useful to make the representation of the rules more accessible.  SQL is a commonly known among developers and database administrators, but it is not a skill typically known by end users.  An effort to evaluate alternative ways to construct the CEP’s rules would be useful.  The system would represent the CEP rule logic such that it could be authored, displayed, edited, and used to produce a targeted CEP system’s rule representation.
Most IoT environments are heterogeneous in nature.  The same fields often have differing names, format, or units of measure.  In order to perform CEP on the generated data, it is useful for it to be standardized prior to processing.  A valuable research effort would be to consider a stateless preprocessing engine that accepted varying known formats and converted to a standardized output for ingestion by CEP.
Fog computing is a computing approach where processing occurs in the device or sensing layer of IoT (Díaz, Martín, & Rubio, 2016).  The motivation behind fog computing is that there are situations where cloud computing is not adequate.  Issues such as latency or location awareness require the processing as close as possible to the device.  CEP could be performed in the fog layer.  This could reduce bandwidth utilization by only sending information when an event was detected. 
IoT communications are comprised of two types of messages.  Sensor data flows from the device to a processing destination.  Data flows to the sensor to cause it to interact with its environment.  For example, a device might report a room temperature.  That data flows through the network and is processed by CEP.  CEP is used to produce actions based on a condition.  If the temperature was over a threshold, a message is sent to the device indicating the fan be turned on to increase the airflow into the room.  An investigation into a framework to integrate CEP output with IoT actuator flow.
IoT is being extended beyond the physical world to include virtual constructs (Ciciriello, Mottola, & Picco, 2006).  This can be viewed as creating an abstraction hierarchy.  The combination of virtual sensors and actuators combined an abstraction hierarchy of virtual sensors should be researched further.

Conclusion

The IoT is a producer of streams of data.  CEP is an efficient and popular way of processing streams of data.  The history of CEP is based on pattern matching and relates to database management systems.  The combination of CEP and IoT is an advantageous one. 

References

Ashton, K. (2009). That ‘internet of things’ thing. RFiD Journal, 22(7), 97-114.
Barnaghi, P., Sheth, A., & Henson, C. (2013). From data to actionable knowledge: Big Data challenges in the Web of Things. Intelligent Systems, IEEE, 28(6), 6-11. doi:10.1109/MIS.2013.142
Bassi, A., Bauer, M., Fiedler, M., Kramp, T., Kranenburg, R., Lange, S., & Meissner, S. (2013). Enabling things to talk: Designing IoT solutions with the IoT architectural reference model: Springer.
Berners-Lee, T. (1989). Information Management: A proposal.   Retrieved from https://www.w3.org/History/1989/proposal.html
Bruns, R., Dunkel, J., Masbruch, H., & Stipkovic, S. (2015). Intelligent M2M: Complex event processing for machine-to-machine communication. Expert Systems with Applications, 42(3), 1235-1246. doi:http://dx.doi.org/10.1016/j.eswa.2014.09.005
Cavalcante, E., Alves, M. P., Batista, T., Delicato, F. C., & Pires, P. F. (2015). An analysis of reference architectures for the Internet of Things. Paper presented at the Proceedings of the 1st International Workshop on Exploring Component-based Techniques for Constructing Reference Architectures, Montreal, QC, Canada.
Ciciriello, P., Mottola, L., & Picco, G. P. (2006). Building virtual sensors and actuators over logical neighborhoods. Paper presented at the Proceedings of the international workshop on Middleware for sensor networks.
Díaz, M., Martín, C., & Rubio, B. (2016). State-of-the-art, challenges, and open issues in the integration of Internet of Things and cloud computing. Journal of Network and Computer Applications. doi:http://dx.doi.org/10.1016/j.jnca.2016.01.010
Gordon, D., & Johnson, M. (n.d.). The story of the trojan room coffee pot: A timeline.  Retrieved from http://www.cl.cam.ac.uk/coffee/qsf/timeline.html
Hui, S., Jiafu, W., Caifeng, Z., & Jianqi, L. (2012, 23-25 March 2012). Security in the Internet of Things: A review. Paper presented at the Computer Science and Electronics Engineering (ICCSEE), 2012 International Conference on.
IDC. (2015). Explosive Internet of Things spending to reach $1.7 trillion in 2020 [Press release]. Retrieved from http://www.idc.com/getdoc.jsp?containerId=prUS25658015
Leavitt, N. (2009). Complex-Event Processing poised for growth. Computer, 42(4), 17-20. doi:10.1109/MC.2009.109
Lin, Y.-F., Wu, C.-W., Huang, C.-F., & Tseng, V. S. (2015). Discovering utility-based episode rules in complex event sequences. Expert Systems with Applications, 42(12), 5303-5314. doi:http://dx.doi.org/10.1016/j.eswa.2015.02.022
Luckham, D. (2002). The power of events (Vol. 204): Addison-Wesley Reading.
Luckham, D. C., & Frasca, B. (1998). Complex event processing in distributed systems. Computer Systems Laboratory Technical Report CSL-TR-98-754. Stanford University, Stanford, 28.
Malamud, C. (1992). Exploring the Internet: a technical travelogue: Prentice Hall.
MarketAndMarkets. (2016). Complex Event Processing (CEP) market worth 4.95 billion usd by 2020.   Retrieved from http://www.marketsandmarkets.com/PressReleases/complex-event-processing-cep.asp
Mehdiyev, N., Krumeich, J., Enke, D., Werth, D., & Loos, P. (2015). Determination of rule patterns in Complex Event Processing using machine learning techniques. Procedia Computer Science, 61, 395-401. doi:http://dx.doi.org/10.1016/j.procs.2015.09.168
Microsoft Windows 3.0. (n.d.).   Retrieved from http://www.oldcomputermuseum.com/os/windows_3.0.html
Oracle. (2016). Oracle Complex Event Processing Retrieved from http://www.oracle.com/technetwork/middleware/complex-event-processing/overview/index.html
Saffo, P. (1997). Sensors: The next wave of innovation. Communications of the ACM, 40(2), 92-98.
Singh, S., & Singh, N. (2015, 8-10 Oct. 2015). Internet of Things (IoT): Security challenges, business opportunities & reference architecture for E-commerce.Paper presented at the Green Computing and Internet of Things (ICGCIoT), 2015 International Conference on.
StreamBase. (2016). StreamBase.   Retrieved from http://www.streambase.com/
Tiburski, R. T., Albernaz Amaral, L., De Matos, E., & Hessel, F. (2015). The importance of a standard security architecture for SOA-based IoT middleware. Communications Magazine, IEEE, 53(12), 20-26. doi:10.1109/MCOM.2015.7355580
Weber, S., Lowe, H. J., Malunjkar, S., & Quinn, J. (2010). Implementing a real-time complex event stream processing system to help identify potential participants in clinical and translational research studies. AMIA Annual Symposium Proceedings, 2010, 472-476.  Retrieved from http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3041381/
Weiser, M. (1991). The computer for the 21st century. Scientific american, 265(3), 94-104.
WSO2. (2016). Complex Event Processor.   Retrieved from http://wso2.com/products/complex-event-processor/

No comments: