Altoros is a 300 people strong consultancy that helps Global 2000 organizations with a methodology, training, technology building blocks, and end-to-end solution development. The company turns cloud-native app development, customer analytics, blockchain, and AI into products with a sustainable competitive advantage. Altoros assists enterprises on their way to digital transformation, standing behind some of the world's largest Cloud Foundry deployments.
Sohrabi, Shirin (IBM T. J. Watson Research Center) | Udrea, Octavian (IBM T. J. Watson Research Center) | Ranganathan, Anand (IBM T. J. Watson Research Center) | Riabov, Anton (IBM T. J. Watson Research Center)
Goal-driven automated composition of software components is an important problem with applications in Web service composition and stream processing systems. The popular approach to address this problem is to build the composition automatically using AI planning. However, it is shown that some of these planning approaches may neither be feasible nor scalable for many large-scale flow-based applications. Recent advances have proven that the automated composition problem can take advantage of expert knowledge describing the many ways in which different reusable components can be composed. This knowledge can be represented using an extensible composition template or pattern. In prior work, a flow pattern language called Cascade and its corresponding specialized planner have shown the best performance in these domains. In this paper, we propose the use of Hierarchical Task Network (HTN) planning for the composition of stream processing applications. To this end, we propose an automated approach of creating an HTN-based problem from the Cascade representation of the flow patterns. The resulting technique not only allows us to use the HTN planning paradigm and its many advantages including added expressivity but also enables optimization and customization of composition with respect to preferences and constraints. Further, we propose and develop a lookahead heuristic and show that it significantly reduces the planning time. We have performed extensive experimentation with stream processing applications and evaluated applicability and performance of our approach.
Kafka Streams is a library designed to allow for easy stream processing of data flowing into a Kafka cluster. Stream processing has become one of the biggest needs for companies over the last few years as quick data insight becomes more and more important but current solutions can be complex and large, requiring additional tools to perform lookups and aggregations. K S in Action teaches readers everything they need to know to implement stream processing on data flowing into their platform, allowing them to focus on getting more from their data without sacrificing time or effort. By the end of the book, readers will be ready to use Kafka Streams in their projects to reap the benefits of the insight their data holds quickly and easily. Bill Bejeck is a Kafka Streams contributor with over 13 years of software development experience.