Stop over-engineering AI apps: just use Postgres

[-]

A_Again@reddit

So in effect Postgres can serve the function of both a noSQL and a vector DB simultaneously? I may have missed it but where is their AI backend code living to do embeddings here?

[-]

Worldly_Expression43@reddit (OP)

That's correct. Pgai is the one doing all the embedding. It's just an extension on top of Postgres. Everything lives within your DB.

[-]

Cachesmr@reddit

not anymore. for some reason, now you need to run a worker. a python worker at that. pgai is now a python library, a huge regression in my opinion

[-]

ZHName@reddit

Yeah I like this more. I was thinking this exact thing about postgres earlier this week.

[-]

yall_gotta_move@reddit

https://github.com/pgvector/pgvector

it doesn't compute embeddings, that's the embedding model's job. it just indexes then and implements fast approximate nearest neighbors search methods.

[-]

Worldly_Expression43@reddit (OP)

pgai is what computes the embeddings

[-]

Worldly_Expression43@reddit (OP)

Yeah if you want to run it locally

It's also available on their cloud

[-]

femio@reddit

Try reading the article.

https://github.com/timescale/pgai

[-]

ttkciar@reddit

Learn it anyway. It will feel pretty natural with some practice, and you'll probably be using it for the rest of your career.

[-]

HarambeTenSei@reddit

Not while mongo coupled with qdranr does most of same thing for a pythonic syntax

[-]

ttkciar@reddit

You assume you will always have that option.

In practice, some jobs will require SQL, and not having SQL skills will mean not being able to apply to those jobs.

[-]

HarambeTenSei@reddit

I already refuse all offers that require SQL and urge them to upgrade to a more modern data management method

[-]

One-Employment3759@reddit

If you're some noob. SQL is universal.

[-]

Worldly_Expression43@reddit (OP)

SQL will never die 😎

[-]

docsoc1@reddit

We built all of R2R inside postgres, if anyone is interested in seeing how we architected - https://r2r-docs.sciphi.ai/

[-]

Previous-Piglet4353@reddit

Heh, I like this development a lot.

Everyone spends 2 years inventing new AI tools and methodologies, or reinventing the wheel (hello design patterns).

And now, the classic tools just extend to integrate with AI instead.

[-]

FormerKarmaKing@reddit

The killer feature of vector databases was always deploying more VC cash.

[-]

Worldly_Expression43@reddit (OP)

Trust me, I've worked for one. On pure compute alone they were burning hella VC cash

[-]

Worldly_Expression43@reddit (OP)

For Postgres it's just another index type, like JSONB

[-]

Previous-Piglet4353@reddit

As it should be, there's nothing particularly unique or special about graphs to warrant a special use graph DB just for vectorizing text.

Every database corresponds to a matrix. Every matrix corresponds to a weighted bipartite graph. It's meant to be simple.

[-]

rorykoehler@reddit

Language specific tooling has come a long way too. Building out a RAG app with Postgres vector atm in Ruby and it was seamless

[-]

Worldly_Expression43@reddit (OP)

Correct. Every data type is just another index

[-]

ehbrah@reddit

Interesting. Tangentially, do you have any experience experimenting e milvus?

[-]

blackenswans@reddit

MariaDB and SQLite also support vectors

[-]

ipokestuff@reddit

yes, don't use this abstraction layer, use another - were you paid to make this post?

[-]

I setup a Postgres db that will automatically vectorize new or changed rows in docker with fast api tunneled out with ngrok so my wife can add/modify entries with ChatGPT with custom actions and recall with vector search. It works great, and wasn’t bad to setup.

[-]

debauch3ry@reddit

What's the tiggering and processing mechanism, if you don't mind sharing?

[-]

DrivewayGrappler@reddit

Yeah, I didn't use pgai-vectorizer—I set it up myself with PostgreSQL triggers and a FastAPI service running in Docker. The process works like this:

Triggering: PostgreSQL triggers detect changes (INSERT/UPDATE) and log them in a small queue table.
Processing: A FastAPI service (in Docker) listens for changes, pulls the affected rows, and:
Embeds the text using OpenAI’s text-embedding-3-large
Stores the embeddings in a separate vector table using pgvector
Marks the processed row as handled

I expose FastAPI via ngrok, allowing my wife to interact with it remotely through ChatGPT’s custom actions, adding/modifying entries and querying via vector search.

[-]

Worldly_Expression43@reddit (OP)

Check out pgai vectorizer. It has a worker that monitors your table and embeds it automatically when changes come in

[-]

debauch3ry@reddit

I assumed the commenter I was replying to was saying "it's so easy I did it myself without pgai". As for pgai, thanks to this post I'm looking at Timescale in general. Employer has me in an Azure estate mind you, but I'm very excited to see MS's DiskANN within easy reach now :)

[-]

Mbando@reddit

I think the larger point that jack of all trade frameworks may be inefficient is interesting.

[-]

HiddenoO@reddit

This shouldn't surprise anybody. The more different use cases a framework needs to support, the less likely it's fully optimized for any single one of them. That's just how it works with frameworks in general, regardless of AI or not.

[-]

InsideYork@reddit

What about emergent intelligence? I think there's been a belief that despite lack of specialization it would be more intelligent if it was larger with different domain expertise.

[-]

HiddenoO@reddit

We're talking about frameworks here, not agents or models. You gain nothing from your framework supporting functionality you're not using, but having to support that functionality may come with trade-offs you wouldn't have to make otherwise.

[-]

Worldly_Expression43@reddit (OP)

Very true esp. with LangChain

[-]

Barry_Jumps@reddit

All the smug engineers who said “you only know python and sql?”.

[-]

sovok@reddit

Yes, when your app needs a relational database anyway, why not do vector stuff in that as well. MariaDB will also get a vector datatype that’s supposedly faster than pgvector: https://mariadb.org/projects/mariadb-vector/

[-]

One-Employment3759@reddit

I lived through the nosql trend and while you can do cool custom DB engines, 90% of the time?

Just use postgres.

[-]

CompromisedToolchain@reddit

Using Postgres for this seems like over engineering :)

[-]

jascha_eng@reddit

Any meaningful app will need something like postgres or similar anyways for all the functionality that's not AI. So why not use it for your embeddings rather than complicating your stack further?

[-]

CompromisedToolchain@reddit

No, that’s not a given. I’ve implemented my own LM (just 38M params) and didn’t contract out the storage to something else. I’ve my own file format based on my needs for sequences, vocab, and training data.

[-]

One-Employment3759@reddit

How do you maintain data consistency during a power failure?

[-]

jascha_eng@reddit

Okay how does a user log in?

[-]

CompromisedToolchain@reddit

Nobody logs in, I run this locally.

[-]

Fast-Satisfaction482@reddit

User accounts don't strictly require a relational database server, but I will soon run into trouble scaling up, if you don't use one. There are VERY good reasons that basically everyone adopted this ages ago.

[-]