After releasing Microsoft Azure Data Factory v2 (ADF) in public preview in September, Microsoft has recently followed up with the announcement of a public preview of new visual tooling for the fully managed cloud-based data integration and ETL service. However, for the September release of the service Visual tooling was not available, making it a manual process to create ADF v2 components and pipelines. The recent release of the visual tooling brings the service more in line with the previous version. The tooling is web-based and is launched in the Azure portal from within the deployed Azure Data Factory. To support copy activities and offloading of computing tasks, there is an additional type of Integration Runtime component that is either Azure-based or Self-Hosted.

After releasing Microsoft Azure Data Factory v2 (ADF) in public preview in September, Microsoft has recently followed up with the announcement of a public preview of new visual tooling for the fully managed cloud-based data integration and ETL service. However, for the September release of the service Visual tooling was not available, making it a manual process to create ADF v2 components and pipelines. The recent release of the visual tooling brings the service more in line with the previous version. The tooling is web-based and is launched in the Azure portal from within the deployed Azure Data Factory. To support copy activities and offloading of computing tasks, there is an additional type of Integration Runtime component that is either Azure-based or Self-Hosted.

Here's an analogy using a concept that we can all relate to: a supermarket. Picture the scene: Shopping list in one hand, shopping basket in the other, you're ready to tackle your weekly shopping in your local supermarket. Your items range from fruit and vegetables to washing detergent, perhaps with some free-range eggs thrown in for good measure. Quite the eclectic mix, but you know that you'll be able to find all you need under one roof. The fact that this is possible is in itself quite remarkable.

Data integration requires merging date from different sources, stored using technologies. Companies build a "data warehouse where aggregated data can be stored and retrieved. This is particularly useful for researchers looking to big data to aid in their investigation and corporations usually during the merging with other companies. Users can access all systems of different sources or interface of web pages but without viewing consolidated data. This organizational level requires particular applications to integrate data.

The release goes on, "Historically, developers targeting gateway and edge devices had to work with multiple databases and data management solutions on each application, which required custom data transformation and integration code for each pair of databases. Compounding the problem, the typical enterprise involves many different types of databases, operating systems and hardware architectures, making a true hybrid data system very difficult to build, deploy and maintain. The result is slower development cycles, high support costs and increased complexity. Actian Zen's technology breakthrough delivers a common data type and file format across a wide range of platforms – from the edge to the gateway to the enterprise – eliminating the need for customized Extract, Transform and Load (ETL) and integration code and enabling developers to focus on creating value-add application features."

The rise of technology has provided publishers with access, for the first-time, to detailed information about trends in the marketplace and the changing needs of their customers. Being able to study and understand consumer interests as they happen is vitally important to publishers in order for them to continue to remain an essential player in the marketplace. Unfortunately, as many companies lament, the sheer volume of data coming in makes it nearly impossible for human analysts to evaluate properly on both large-scale and granular levels.In order to better understand and respond to the results of this information dump, publishers must rely on an automated system that can analyze the data and provide insights that better inform human decisions on what type of content to publish, how and when to publish, and even where to release this content. The best way of handling the analysis of this information is through machine learning software, made famous by IBM's Watson. How machine learning works is the software is provided with a model or set of functions desired by the creator and, as the software performs that set of functions, it learns from the data it is analyzing how to better perfect the model.

Some predict that Artificial Intelligence will drive the next industrial revolution. What is certain is that over the next few years AI will become more important to marketers. But to unlock AI's huge potential you need an AI strategy. Here are five Ps to help you develop yours. How can AI help your organisation? What business problem are you trying to solve?

See Ben Lorica's video "Trends in AI, Data Science, and Big Data" on Safari for a recap of research initiatives and movements in 2017. Here's what we expect to see--or see more of--in the data world in 2018. Graphs and time series have been a crucial part of the explosion in big data. These new analytic and visualization tools will help product groups devise new offerings, especially for use cases in security and fraud detection. In 2016, I started hearing companies express interest in data sharing platforms, and startups have now begun to build data exchanges to allow companies to share data across organizational boundaries, while protecting privacy and IP.

A recent Forbes article on the 10 Predictions for AI, Big Data, and Analytics in 2018 states that Data engineer will become the hot new job title, displacing its sibling role of Data Scientist. Gil Press goes on to write that Indeed.com Intrigued, I looked at the job descriptions of Data engineer job postings by leading data-driven companies like Amazon and Facebook on LinkedIn. Strong Data Warehouse skills with a thorough knowledge of Data Extraction, Transformation, loading (ETL) processes and Data Pipeline construction expertise stood out as the essential and basic qualifications of an ideal Data Engineer. Who is a Data Engineer Anyway?

This hands-on, 5 half-day Live-Online Spark 301 training targets experienced Data Scientists wishing to perform data analysis at scale using Apache Spark. The class will be held on the following days and times: Monday, January 22 to Friday, January 26, 2018, from 7:00am to 11:00am PST each day. This course covers an overview of Apache Spark, hands-on projects utilizing extract- transform-load operations (ETL), employing exploratory data analysis (EDA), building machine learning models, evaluating models, and performing cross validation. This course covers the same material as our three-day, in-person Apache Spark For Machine Learning & Data Science (Spark 301) course. All hands-on labs are run on Databricks Community Edition, a free cloud based Spark environment.