Building a recommendation engine inside Postgres with Python and Pandas
Just because you can do something doesn't always mean you should. Embedding all of your application logic directly in the database can make tracking migrations and releases difficult. At the same time, a complex pipeline that takes a nightly extract, loads something into Spark, generates results, that you then feed back into the database isn't exactly lightweight. In the case of plpython3u and pandas, scheduling something like the above to run daily with pg_cron could be a much simpler solution. With a mix of SciPy, NumPy and Pandas there is a lot of interesting potential here and I'd love to hear what practical uses others come up with @crunchydata, or give it yourself a try-our database-as-a-service Crunchy Bridge comes already preconfigured with plpython3u and SciPy, NumPy, and Pandas.
Jan-10-2022, 21:36:55 GMT
- Technology: