A new study commissioned by Trifacta is shining the light on the costs of poor data quality, particularly for organizations implementing AI initiatives. The study found that dirty and disorganized data are linked to AI projects that take longer, are more expensive, and do not deliver the anticipated results. As more firms ramp up AI initiatives, the consequences of poor data quality are expected to grow. The relatively sorry state of data quality is not a new phenomenon. Ever since humans started recording events, we've had to deal with errors.

The data type, missing data and, anomalies can be defined in broad terms as follows: The data type is the common characteristic that is expected to be shared by entries in a column, such as integers, strings, IP addresses, dates, etc., while missing data denotes an absence of a data value which can be encoded in various ways, and anomalies refer to values whose types differ from the given column type or the missing type. In order to model above types, we have developed PFSMs that can generate values from the corresponding domains. This, in turn, allows us to calculate the probability of a given data value being generated by a particular PFSM. We then combine these PFSMs in our model such that a data column x can be annotated via probabilistic inference in the proposed model, i.e., given a column of data, we can infer column type, and rows with missing and anomalous values.

Machine Learning and AI are all the buzz. In the last year, IDC reports that 37.5 billion dollars will be spent on machine learning and AI investments, increasing to close to $100 billion by 2023. Yet organizations still struggle to get value out of their machine learning and AI investments. It's widely known that 80% of any data science project is spent wrangling the data. To compound this fact, machine learning and AI models require high quality data in order to be effective.

In the past few weeks, two important developments in artificial intelligence research have gone largely unheralded. Both hint at just how earth shaking – or at least industry-shattering – A.I.'s potential really is. The first item was news that a Hong Kong-based biotechnology startup, InSilico Medicine, working with researchers from the University of Toronto, had used machine learning to create a potential new drug to prevent tissue scarring. What's eye-popping here is the timescale: just 46 days from molecular design to animal testing in mice. Considering that, on average, it takes more than a decade and costs $350 million to $2.7 billion to bring a new drug to market, depending on which study one believes, the potential impact on the pharmaceutical industry is huge.

Telstra's independent venture capital arm has shown its intention to expand into the artificial intelligence data market following a $US100m (145m AUD) capital raising for San Francisco company Trifacta. Trifacta employs machine-learning technology to deduce a greater depth of insights from the increasing level of data migrating to cloud-based storage. Australia's largest venture capital fund, Telstra Ventures Fund No 2, led the investment, joined in the round by the likes of Energy Impact Partners, NTT Docomo, BMW Ventures and ABN AMRO. Telstra Venture joins a long and credible list of existing investors from Accel Partners, Greylock Partners, Ignition Partners and Google. "The share register for Trifacta is very impressive. It is great to have so many experienced and impressive co-investors in this deal. That is a really massive plus for us," Mr Koertge said.

Trifacta's software employs machine–learning technology to assist data scientists derive faster insights from the increasing levels of data migrating to …

The insights offered by Big Data are key to many businesses today. Getting the information that's hidden within it isn't easy but there are plenty of companies set up to help organisations do just that. Trifacta is one such company. It specialises in cleaning and preparing data, ready for it to be mined for key information, or train machine learning algorithms. One of its key products, Cloud Dataprep, makes use of Google's large cloud infrastructure footprint, which puts the data preparation tool in the hands of all manner of companies.

At its Google Cloud Next conference in San Francisco back in March, Google unveiled Cloud Dataprep, a service that lets companies clean their structured and unstructured datasets for analysis in, for example, Google's BigQuery, or even for use in training machine learning models. Over the past six months, Cloud Dataprep has been in private beta, but Google is now officially graduating the service to public beta for anyone to use. Some reports indicate that analysts and data scientists can spend up to 80 percent of their time cleaning and preparing raw data for analysis. This is where Dataprep comes into play, as it can automatically detect data type, schema, and even where there is mismatched or missing data. A key facet of Dataprep is the visual layout, which makes it easier for people who aren't data engineers to alter or add to their datasets.

Above the Trend Line: machine learning industry rumor central, is a recurring feature of insideBIGDATA. In this column, we present a variety of short time-critical news items such as people movements, funding news, financial results, industry alignments, rumors and general scuttlebutt floating around the big data, data science and machine learning industries including behind-the-scenes anecdotes and curious buzz. Our intent is to provide our readers a one-stop source of late-breaking news to help keep you abreast of this fast-paced ecosystem. We're working hard on your behalf with our extensive vendor network to give you all the latest happenings. Be sure to Tweet Above the Trend Line articles using the hashtag: #abovethetrendline.

The latest version of Alpine Data's analytics platform seeks to combine data with machine learning for business users by shifting the focus from algorithms while adding human collaboration and governance capabilities to machine learning projects. San Francisco-based Alpine Data said Wednesday (May 25) its Chorus 6.0 analytics platform targets a wider swathe of the business analytics spectrum by accelerating "the delivery of data into action and to create a clear, repeatable process for continuous business improvement." Among the reasons that heavy enterprise investment in big data projects often does not pan out, the company argues, are the often-fruitless search for the "right" algorithm. Another pain point is enterprise struggles over data infrastructures and governance. Data governance refers to the overall management of data access rights and other security considerations.