Collaborating Authors

software engineering

Towards ML Engineering: A Brief History Of TensorFlow Extended (TFX)


Software Engineering, as a discipline, has matured over the past 5 decades. The modern world heavily depends on it, so the increased maturity of Software Engineering was an eventuality. Practices like testing and reliable technologies help make Software Engineering reliable enough to build industries upon. Meanwhile, Machine Learning (ML) has also grown over the past 2 decades. ML is used more and more for research, experimentation and production workloads. But ML Engineering, as a discipline, has not widely matured as much as its Software Engineering ancestor. Can we take what we have learned and help the nascent field of applied ML evolve into ML Engineering the way Programming evolved into Software Engineering? In this article we will give a whirlwind tour of Sibyl and TensorFlow Extended (TFX), two successive end-to-end (E2E) ML platforms at Alphabet. We will share the lessons learned from over a decade of applied ML built on these platforms, explain both their similarities and their differences, and expand on the shifts (both mental and technical) that helped us on our journey.

Software Engineer, iTunes Big Data Social - IoT BigData Jobs


The iTunes big data engineering team is looking for talented server-side engineers to build and enhance social features such as those underpinning Apple Music. This is your opportunity to contribute to key Apple services built using massively scaled systems, on a team located in San Francisco and working closely with Cupertino and London. Key Qualifications Minimum of 5 years professional software engineering experience. Proficiency in building Node.js applications. Experience with building RESTful APIs. Experience with a NoSQL solution, document store, or key-value store (e.g. Cassandra, Redis, MongoDB, Couchbase). Comfortable with Linux command line tools and basic shell scripting. Description Our team is responsible for architecting and delivering services such as those central to Apple Music Connect that allow users and artists to interact with each other. To build these features, we create server-side applications that employ a combination of microservices, message-passing, caching layers, and distributed databases. We serve our data over cleanly designed RESTful HTTP endpoints used by multiple client platforms making a massive number of requests per second at millisecond response times. This is a great opportunity to join a small but growing team of motivated engineers, with wide responsibility and high-profile feature ownership. Whether you’re interested in architecture, data modeling, plumbing data pipelines, or designing endpoints, there are numerous possibilities for building new features from scratch and enhancing the existing infrastructure. Education Education: BS or MS in Computer Science, or equivalent experience Additional Requirements Experience with building highly scalable services using a microservices architecture. Experience with message-based architectures using Kafka or other another message broker. Experience with Agile software development methodologies including Scrum and TDD (test-driven development). Ability to collaborate with cross-functional teams. Familiarity or experience with Java or another object-oriented programming language. Experience with Git.

Managing the Organized Chaos That Is Software Development


Disclaimer: This article is written mostly by GPT-3 given the first paragraph as a prompt; a few edits were made for style and clarity. Complex software projects require a level of discipline to ensure meeting deadlines and hitting milestones on time; however, software development is a creative process as well that calls for flexibility and leeway for experimentation. Software organizations need to strike that balance well to stay innovative and effective at the same time -- you may call it organized chaos. Software development projects can be chaotic, especially if you don't know what to do. If you ask any software developer about their experience, you'll likely hear one of two things: "I have no management skills," or "I don't have enough time to manage my team."

Unpopular Opinion – Data Scientists Should Be More End-to-End - KDnuggets


Recently, I came across a Reddit thread on the different roles in data science and machine learning: data scientist, decision scientist, product data scientist, data engineer, machine learning engineer, machine learning tooling engineer, AI architect, etc. It's difficult to be effective when the data science process (problem framing, data engineering, ML, deployment/maintenance) is split across different people. It leads to coordination overhead, diffusion of responsibility, and lack of a big picture view. IMHO, I believe data scientists can be more effective by being end-to-end. Here, I'll discuss the benefits and counter-arguments, how to become end-to-end, and the experiences of Stitch Fix and Netflix. I find these definitions to be more prescriptive than I prefer. Instead, I have a simple (and pragmatic) definition: An end-to-end data scientist can identify and solve problems with data to deliver value.

Solving the AIOps, DevOps, And ITSM Conundrum -


Quickly shifting to remote work has enterprises looking to meet the ops needs of a suddenly distributed team, and there are open source options to get them there. The recent mad rush to scale to remote work may prove to be a key chapter in DevOps and AIOps evolution. This need for rapid, widescale change is creating a real conundrum concerning AIOps, DevOps, and ITSM, as organizations seek the best monitoring and incident response solution for their now distributed enterprises. The key question both the DevOps and IT service management (ITSM) communities need to answer is how quickly they can pivot and adapt to increasing demands for operational intelligence. Artificial intelligence for IT Operations (AIOps) brings together artificial intelligence (AI), analytics, and machine learning (ML) to automate the identification and remediation of IT operations issues.

Webinar: Grady Booch: Software Architecture for AI-intensive Systems


PLEASE NOTE THAT THIS WEBINAR WILL START ON WEDNESDAY, 16 SEPTEMBER, 2020, AT 6:30 PM ***BST (LONDON TIME)*** (1.30 PM ***EDT (NEW YORK TIME)***) TITLE: Software Architecture for AI-intensive Systems ABSTRACT The problem at hand is partly the application of software engineering best practices to AI, but more so the evolution of software engineering to attend to software-intensive systems that contain AI components. BIOGRAPHIES Grady Booch is Chief Scientist for Software Engineering at IBM Research where he leads IBM's research and development for embodied cognition. Having originated the term and the practice of object-oriented design, he is best known for his work in advancing the fields of software engineering and software architecture. A co-author of the Unified Modeling Language (UML), a founding member of the Agile Alliance, and a founding member of the Hillside Group, Grady has published six books and several hundred technical articles, including an ongoing column for IEEE Software. Grady was also a trustee for the Computer History Museum.

10 Predictions On Software Development Trends Of 2022


What are the trends in software development over the second half of 2020? This is like no other year. The prevailing issue has turned the world upside down, pushing companies to take on new technology's challenges and analyze their digital strategies. Digital has become the principal (and, in some cases, only) channel of customer interaction and engagement. Enterprises with the digital projects designed to be implemented within the next one to three years need to speed up their initiatives.

Software developers: How plans to automate coding could mean big changes ahead


For the vast majority of humans, writing code is akin to learning a new language – but researchers from Intel and MIT are on a mission to change that. And the solution they are coming up with is to build code… that can code. Called machine programming, the field that the researchers are looking at is concerned with automating software development. And the team has just revealed a new tool that takes developers one step closer to the prospect of, one day, having machines that can program themselves. MISIM (Machine Inferred code Similarity), the new technology invented by Intel and MIT's labs, effectively studies snippets of code to understand what a piece of software intends to do.

Top 5 Mobile App Development Trends to Watch Out for in 2020


Mobile Development has become one of the most critical aspects of many companies. This is due to the acquisition of a more organic base of clients. Without a mobile-optimized solution, a company could face the question of lagging behind its competitors. Given that revenue from mobile apps' development has reached a new peak, people are following the trend in app development. Both the users and the developers follow the path of making life more comfortable.