A traditional software company releases its flagship product maybe every few years. Each release can include hundreds of new features and improvements. Because releases are infrequent, users can grow impatient waiting for each new release and are thankful when it finally arrives. Disappointment sets in, however, when bugs are found and features don't work as expected. Under great stress and with great turmoil, an emergency release is produced and put into production (hurried through the regular release process, often achieved by skipping tests), which has still more bugs, and the process repeats with more emergency releases, leading to more frustration, stress, and disappointment. Worse yet, new business opportunities are missed or ignored because of doubt, uncertainty, and distrust in the IT department's ability to deliver value.
Years ago, prior to the advent of Agile Development, a friend of mine worked as a release engineer. His job was to ensure a seamless build and release process for the software development team. He designed and developed builds, scripts, installation procedures and managed the version control and issue tracking systems. He played a mean mandolin at company parties too. The role of release engineer was (and still is) critical to completing a successful software release and deployment, but as these things go, my friend was valued less than the software developers who worked beside him.
As DataOps continues to gain exposure, people are encountering the term for the first time. Below is our list of the most common questions that we hear about DataOps. The best way to explain DataOps is to review its intellectual heritage, explore the problems it is trying to solve, and describe an example of a DataOps team or organization. Our explanations below start at a very conceptual level, but then quickly proceed into pragmatic and practical terms. We find this is the best way to help data professionals to understand the potential benefits of DataOps.
Are we holding true to the principles of Agile, as described in the manifesto? Are we finally "uncovering better ways of developing software by doing it and helping others do it"? Are we finally valuing "individuals and interactions over processes and tools"? Maybe -- everyone is trying their best. But we do seem to be more sprint-like in our delivery, delivering working software frequently, "from a couple of weeks to a couple of months [2020 update -- a couple of hours], with a preference to the shorter timescale."
The face of modern IT has been changing swiftly, as data centers transition from physical infrastructures to virtual infrastructures. 'Cloud' applications extensively use microservices (small chunks of independent functions), deploy via containers and communicate via APIs. A key driver for this transition is the increasing need for rapid development and deployment of cloud-native applications, as businesses actively adopt digital strategies - either to disrupt their markets, or to simply remain competitive. This has led to a new practice of combined application development and IT delivery called DevOps. DevOps (a portmanteau of "DEVelopment" and "OPerationS"), refers to a set of practices that encourages a handshake between software development and Operations.