GPT4All - The better AI chatbot?

[Bitblazer]

My goal is give a document manual and let it to answer questions about it.
I think this way would be easiest
I don't know where to start, but I will search about.

The NOMIC gpt4all discord chat is definately a place where you should keep an eye on. Just recently somebody else asked how to do that. I am currently creating a GUI based tool to generate own models. In a few months, we should all be able to talk with the AI about purebasic and your product with the help of special AI model files.

On top of that, i hope in some months we all will be able to use the AI from inside any purebasic software.

[Caronte3D]

Thanks for the info

[idle]

Nice! Works now for me
It understand only English or depend of the model loaded?

EDIT:
Yes, can respond in another languages if you ask for it, but very bad

That's good to hear. The current example needs some work.

[Caronte3D]

One of the coolest areas of this topic is to be able to use gpt4all with your owns documents so you can ask questions about that documents, manuals, etc...
Take a look to this video, maybe we can use it from PB?...

https://www.youtube.com/watch?v=l-Ji31HlC68

[idle]

It's a bit like gopher on shrooms, in that it hallucinates the answers. Could be useful.

[Caronte3D]

It's a bit like gopher on shrooms, in that it hallucinates the answers. Could be useful.

Take a look at this website, this service does a very good job and not hallucinate (or almost):
https://www.chatpdf.com/
I'm on the waitinglist for their API release, but I'm searching alternatives.

[idle]

It'd be really useful in academic literature searches, or comparing many versions of a text.

I'm still floundering on understand the models and finding how to train them in c/c++

[Kiffi]

Take a look at this website, this service does a very good job and not hallucinate (or almost):
https://www.chatpdf.com/

Impressive!

I uploaded a small German PB tutorial (the PB help is unfortunately too big for the free account) and my questions ("show me the syntax of openwindow()", "what is the eventloop?", "show me the code of creating a button", "how can i force the declaration of variables?") were answered correctly.

even this question was answered correctly:

what wrong with this code?

Code: Select all

OpenWindow(0, 0, 0, 400, 200, "Button Example", #PB_Window_SystemMenu|#PB_Window_ScreenCentered)

Repeat
  Select WaitWindowEvent()
    Case #PB_Event_CloseWindow
      Break
  
Forever

CloseWindow(0)

The code you provided is missing an "EndSelect" statement inside the event loop. This will cause a compilation error when you try to run the code.

Here is the corrected code with the missing "EndSelect" statement added:

Code: Select all

OpenWindow(0, 0, 0, 400, 200, "Button Example", #PB_Window_SystemMenu|#PB_Window_ScreenCentered)

Repeat
  Select WaitWindowEvent()
    Case #PB_Event_CloseWindow
      Break
  EndSelect
Forever

CloseWindow(0)

In this corrected code, the "EndSelect" statement is added after the "Case #PB_Event_CloseWindow" statement inside the event loop. This ensures that the Select/Case statement is properly closed and the code can be compiled and executed without errors.

Note that the corrected code still only handles the CloseWindow event and does not include any other event handling code. You can add additional cases to the Select/Case statement to handle other events as needed.

[Caronte3D]

Yes, it's impressive indeed.

[idle]

I'm still trying to see where to poke the stick. I found the document scanning stuff in the gpt4all chat program but I still can't see where it loads it into the model and they say procedural programing is spaghetti.

[Bitblazer]

I'm still trying to see where to poke the stick. I found the document scanning stuff in the gpt4all chat program but I still can't see where it loads it into the model and they say procedural programing is spaghetti.

Same, it seems to be crucial to not overflow some internal (index?) array, so i was told that choosing the tokens will make the difference between good and bad modules. Throwing just some directories with PDF files or websites at the AI is possible, but tends to end with mediocre to bad modules.

I am still investigating further on it. Maybe at some point we will just start with a mediocre module and finetune it later, while adding web/discord interfaces.

[Bitblazer]

It seems to me that the fastest and freely usable AI model type with a open license that is compatible with gpt4all, is a MPT library. The other two model types have legal problems or the ML training ecosystem seems to be currently worse.

A company that offers a commercial training service resulting in MPT models is MosaicML.

This is their companies organisation card at the huggingface AI platform.

This video shows how the training works with the MosaicML cloud and their service.

Check out the blog post about their 30b base model. The post includes prices for fine tuning of their 30b based model.

I did some tests for fun on their MosaicML MPT-30B-Chat.

Their prices are the most affordable ones that i found and their resulting MPT library is technically compatible with GPT4All.

I did mirror the Purebasic documentation and the Tips & Tricks forum already and have them ready to be used for fine tuning of our own purebasic MPT model.

The pro:
- your own ai based chat bot help. How cool is that? A (currently) unique technical feature that clearly says that a product is modern.
- you could use the resulting MPT model for all kind of services based on GPT4All. A web front end, a discord bot.
- an AI example button for each command. The button would do a request like "show a documented example of using the XYZ command in purebasic" to feed it into a MPT-30B enhanced MPT based AI. That already works without a specially trained MPT library, but with mediocre result quality.

The contra:
- it still costs a few thousand bucks
- the resulting AI does hallucinate. For example in my tests with their raw 30b model, the AI happily included .NET libraries into the answers.

Is it worth it?

I honestly don't know. I can't judge the positive marketing effect for purebasic as a product, the final training costs might be low, but could still end up as being wasted money. The prices will come down - over time - with future GPU cards delivering more power for less energy, but at the same time, other competitors will use AI too.

[Bitblazer]

Check out the plug-in interface and the LocalDocs Plugin.

The combination of gp4all and the LocalDocs Plugin allows everybody to run a local ai for free on local confidential documents. That keeps your information private and still allows you to use an AI to create summaries, overviews and act like a search and indexing tool on steroids.

Exactly what companies and privacy aware people wanted.

[idle]

I've rebuilt the libs to latest version, hopefully I have all the dependencies.

windows x64
https://dnscope.io/idlefiles/gpt4all_pb.zip

you need to download the models from
https://gpt4all.io/index.html
I tested

"ggml-model-gpt4all-falcon-q4_0.bin"
"GPT4All-13B-snoozy.ggmlv3.q4_0.bin"

you can also use compatible models from
https://huggingface.co/models

save the model to the source directory and change line 272 in gpt4all.pbi run it and browse to 127.0.0.1

[Caronte3D]

libunwind.dll not found