Encoding fixed length high cardinality non-numeric columns for a ML algorithm
ML algorithms work only with numerical values. So there is a need to model a problem and its data completely in numbers. For example, to run a clustering algorithm on a road network, representing the network / graph as an adjacency matrix is one way to model it. Similarly, a tabular data with a mix of numerical and non-numerical / categorical data also needs to be transformed or encoded to a table of only numbers for a ML algorithm to work on. Columns of string values are quite common in tabular data and in this article, some ideas on how to encode them, especially ones with high cardinality and are of known lengths like IP addresses, mobile numbers etc. are discussed.
Aug-8-2018, 23:53:53 GMT
- Country:
- North America > United States
- New York (0.06)
- California > San Francisco County
- San Francisco (0.05)
- North America > United States
- Technology: