Vibecoding

Admin

Administrator
Moderator
Messages
4,389
#1
I recently got into vibecoding because i wanted to get an accurate score distribution for my favorite poker variant.

It's actually a really nice game once you understand the rules so i wanted more people to be able to try it. I recently expanding the evaluator to cover all variants of it and so far i have not found it to make a single error.

evaluator-NLP8.png


It actually started with me simply wanting to get an accurate distribution over the scores and i did get what's hopefully going to be an accurate distribution via chatGPT and claude (i asked bot to make a sim so i could compare results and hand evaluations). The distribution i got ended up being nicer than i expected it to be.

score-distribution-100M.png


Later i made a version where you play against 3 bots but that did require me to provide a lot of guidance due to LLMs having a poor understanding of poker theory. I did however not see any need to make the bots super-good since the goal was to let people easily try out the game and for that purpose they are already good enough even if they are fairly easy to get an edge against.

https://nlp8.vintologi.eu/

Main thing that remains is to get a working sim for trying out the profitability of various starting hands (when the mediocre bots play them) but i am already quite satisfied with what i have been able to achieve just using free AI (i did however have to jump a bit between models due to free plans running out and due to some models not being able to do what i asked of them). I think i will need to redo the post-flop model from stratch since the model made by the AI was quite bad (that became very clear when i looked at the actual code).

People like to hate on AI but there is no way i would have been able to make this without it without spending weeks/months learning programming (especially not considering i had to both make C++ and HTML/javastript versions.

I did do some manual editing of the code on my own (such as to put links where i wanted them and to adjust MDF weights for the defense against all-in shoves).

One thing that did help was to have a detailed PDF explaining the poker variant (including examples) since that helped the AI models understand what i meant and also gave it examples to test it against. Still it did get some stuff wrong which i ended up having to ask it to correct later.
 

Admin

Administrator
Moderator
Messages
4,389
#2
AI has a tendency to take do a shoddy job to produce output instead of doing things properly
You end up having to direct it over and over again to actually do things in a more proper manner.

So if you rely on AI and do not check what's it actually doing you end up with loads of shitty code and you get stuck with very sup-optimal software where there is no good way to improve it further because the entire things is broken and dysfunctional to the core (to the point where you end up having to start from scratch.

Often when you tell AI to do something it will not actually do as instructed and instead do something significantly worse and you might not actually notice that unless you actually examine the code to see what it actually did (the software may seem to work ok at first but then you will notice that things don't quite add up).

I earlier got the impression that chatGPT had done some good empirical work to develop a decent preflop evaluator only to later discover that the model it made was awful and then when i tried having it improved the model it really didn't want to do things properly and it just kept doing stuff wrong. I asked it to change the model to make sure it aligned with earlier all-in equity measured (against 2 random hands) but it didn't actually align it properly and admitted that after i told it to examine the model.
 

Admin

Administrator
Moderator
Messages
4,389
#3
Very often when an AI claims to have fixed some issue the issue isn't actually fixed
They make you think the problem is solved but then you run into the same or similar bug again or you discover that the supposed fix introduced more problems than it solved.

This is in line with the general theme when using LLMs for coding. You have to constantly watch what they are doing and direct them, especially if you are not using a top tier model. It's like having a dishonest employee that will try to make you think they are working for you but that will constantly cheat and do dishonest things in general behind your back.

I had chatGPT claim to have fixed a problem 5 times only to then find out that the problem wasn't actually fixed. That's pretty bad if you trust the AI to code something of actual importance (you might think it fixed some problem only to later discover it just made things worse).
 
Top