LLMs: Where I see this technology going forward
So it's been no secret that I've been diving into AI and LLMs, which are really one in the same from my point of view, an interchangeable set of characters which describes this technology. I've been learning how this technology works over the past several weeks, and with each day, evolving to a new set of interesting research and tests. Yesterday was a very long day for me, as I created several different local AI applications, which do not use any cloud/data centers. My goal here is to determine how well a local model can perform on decently modest hardware. I am running these local models on a gaming laptop with a hybrid GPU, a combination called Optimus which works decently well on Linux. Without an external monitor connected, my Linux desktop on the internal LCD screen uses the Intel Integrated Graphics, which as a lot of Linux vets know, works amazingly on the Linux desktop, even when compositing is involved, it's very stable. This frees any other process on my laptop to use the GPU in a dedicated fashion, which is where my local models run. The replies back from my local model are almost just as quick, if not the same speed as a cloud model.
It is also able to provide me with a lot of useful information without any Internet connectivity, it has the odd mistake, as yesterday it was unaware of who the late Gary Kildall of Digital Research was, and thought he died at the age of 77 in I think it mentioned 2022 or something, which is completely wrong... So, the local models do seem to struggle with more vague historical information that not many people might care about, like who really has casual conversations about either Gary Kildall or Digital Research in 2026 unless they were doing specific historical research on the subject. When performing any research using a local-only model, always confirm that the local model has enough data on your research subject before continuing. I'd recommend asking it a more general knowledge question about the subject, something which you already know personally, and if the local model comes back incorrect, then don't base any additional research using that local model. Perhaps connect the local model to the Internet so it can do it's own research for you, use a different local model, or if your research is very niche, then perhaps even use a cloud model.
Less Chat; More Work
One thing that I believe makes especially a local model less reliable is the quality of the prompts involved. If the computer operator does not use an optimum prompt to get a specific task done, then it will be more prone to errors, and other known AI issues that people notice. I hate to say that in these situations where an AI might make a huge mistake, it probably came down to not being specific enough in their prompt, leading the AI to assume certain details which the computer operator left out. Never let an AI assume, always give it everything it should need to know, to ensure that it can properly follow all of the directions you have given it. This also includes having a proper System Prompt configured before you even begin to tell it about the task you are working on. The System Prompt sets the stage for the AI, giving it explicit guidelines to follow, much like if you were a movie director and advising an actor of how to take their role, this is essentially what the System Prompt is, and does. Once the AI has their role, then you can interact with it via standard prompts and questions, much like if the actor were now a part of the film, and playing their part for the cameras. This is the best way to think of how AI works in my mind, you are the director, and you need to be very explicit on what you actually want your actors to do during filming. There is no script, you are an improv director, where you then ask the AI to perform on the spot using those initial guidelines you set.
As not everyone has the abilities to create such prompts, and ask the proper questions, this is where AI focused applications will come in. Rather than these applications providing any chat interface, it might for say an expert mode, but in a regular end-user mode, a low-level chat interface will not be exposed to the end-user. But, rather, the application will be presented just as any other application would be. The end-user would select options as they would normally.
For example, let's take a AI enabled database application from an end-user point of view. The end-user opens the database program, it will appear as they do today, be familiar and understandable to any current database user to navigate and use. However, under the hood, many of the standard functionality which may have been done using traditional programming, might have been updated to now use an LLM. Let's take the simple idea of converting data from one format to another. Let's say this end-user has a huge legacy spreadsheet, and they need to convert it into a proper database with several tables, with relational links between them. The end-user would open a standard file open/import dialogue box where they would specify the file they want to convert, and perhaps even some additional details in plain English, such as I want these specific columns to be separated into it's own table, duplicates removed, and then linked back to the entries in the main database. Upon pressing the Process button, the application will spin up the local model needed to perform the task, configure an appropriate System Prompt to ensure the data is processed correctly, then provide both the data from the spreadsheet, along with a prompt created by the application, and combined with the formatting line which the end-user provided. At this point, the local model can do several different things. It can either just return whatever information in a chat response, or, the AI enabled application provides it's own tools to the model, which can allow it to create new databases, new tables, new functions, etc... This way, the AI can process the entire request within the database application using tools provided by the database application itself. In this case, there would never be any need for more riskier tool calling, such as command-lines, or running full programs. In this situation, all of the tools the AI model needs to complete it's request are entirely provided by the database application itself.
This situation frees the end-user from trying to overly think up both an appropriate system prompt, a properly formatted question, and then hoping that it works correctly. Creating AI applications like this, can make AI much more productive, less trial and error, and just getting right down to being immediately productive without any second thoughts. It should be the application developers whom think of System Prompts and who communicate to the local application AI model. As a development studio would have more resources and time to put into this sort of research, and they would also know their own application inside and out, allowing their developers to ensure that the System Prompt and any requests being made are essentially fool-proof. These application developers can then decide if a expert mode which can allow end-users to actually chat with the AI should or shouldn't be available. I don't think a full chat interface is always required in every AI application, and these sort of chat interfaces should be provided to end-users more sparingly, as the wrong prompts can lead to more problems than solutions.
AI won't take THAT many jobs
I can see the potential for AI use in a lot of industries, but trying to solely depend on AI, and it's chat or prompting interfaces is fool-hearty and will only lead to issues.
Humans will always value the creativity of other humans, it's just in our very nature. Sure, a creative story an AI might provide would be interesting to read on passing, but I feel a lot of people will still appreciate stories written by real human beings the most.
In a more recent AI experiment I created, I made the personalities of various people, real and fictional to see what sort of interesting conversations they could have together. I created a somewhat believable exchange between Linus Torvalds and Bill Gates, which I think turned out rather decently as a showcase of what this technology can do. I also created a fun exchange between Dorothy and Rose from The Golden Girls TV show, which turned out really well, and felt like parts of it could have been in a real episode.
Based on this experiment, here are my thoughts, while I believe that this could be used to assist in brainstorming new ideas for a project, or even a TV show or movie, it will never fully take over the role of a writer. This sort of program I wrote could for example be used in a writer's room to assist with brainstorming, where they place the entire description of their characters into a model, and then have them chat with each other about an input subject. The writers would then analyze the chat conversation between their two characters, which should provide the writers with some new ideas for those characters which they may never have explored before. The writers would never take an exact character conversation, but rather use it for their own character research to determine where these characters should go next in their TV show or movie. I feel this could be a great use of this technology within the entertainment industry.
I see an AI model as being a partner in research, rather than a force which will eventually replace the human race. We as a species should stop thinking in binary, as that is the largest issue I see with humanity, is it's instinctive nature to always see everything from a binary point of view. If AI is approached more as a super high-tech research assistant, then that changes how we perceive what AI actually is, and how it can shape our future.
How We Should View AI
At this moment in the time, the most end-user, and consumer facing part of AI is the chat and prompt interfaces. Where people are able to chat with these AIs as though they were their own personal assistants, or to use English language prompts to generate images and videos. This is how the current public is exposed to AI, and now it makes sense why some people are a bit concerned about this technology and that it might begin to replace them. It generates very human-like conversational responses, which has stirred up a lot of issues, mostly from those who may not fully understand it. I cannot say that I fully understand how it is able to generate what it can, but from what I'm beginning to understand, is that it is merely just a super advanced text completion system, where it will determine if the next word properly aligns with the current context based on various weights. Without going into too much technical detail, here is how it essentially works from a very simple non-technical description. Each word in the human language has relations to other words. Some words are further apart from each other, such as words which have zero relation to one another, take a skateboard and a fridge. A skateboard would be close to words like sports, extreme, ollie, and other similar terms. So, when the model is talking about this subject, it's weights will be stuck using similar words, so it won't just start talking about fridges out of nowhere, and keep the words being produced as close to the subject matter as possible. Now, as you could imagine, when we have a sentence, or large paragraph, it will build up a larger context, and include more words, some more similar to skateboard, and some which may not be as closely related, such as a festival. So, if you bring up a sentence talking about skateboards and festivals, even though they aren't entirely related, the AI model will be-able to begin generating more words based on both of these subjects and begin talking about skateboarding festivals, rather than just words close to skateboard. Each word essentially has a weight against another word, some are much further apart, and so those words won't be a part of the generated text, but words closer to the subject will have higher weights, and thus will be a part of the generated text. Adding more context will only give the AI more words to pull from, as now festival is close to a lot of words, which skateboard by itself isn't a part of, allowing the AI to build upon that newly added context. Since words are attached in the dataset within the model to a lot of different subjects, such as real world events and facts, rather than just being a plain dictionary, the model will always be able to see topics related these words depending on the context. Asking about skateboarding festivals, and if the AI model's dataset has any actual articles on this, it may bring them up if it feels like it would be relevant to the current discussion. I also say feels not on the way that we feel, but based on the current Temperature, and related context. If the Temperature is set higher, than it will be more likely for the AI to generate text which might not be entirely related, and not as predictable. To keep a model completely on track with your exact context, then set the Temperature low, even 0.0 to prevent it from getting off-track, and keep focus on just the context given.
That should hopefully clarify how this technology sort of works, I know it's probably not 100% accurate(except for the Temperature setting mentioned), as I'm still learning about how it all works, but at this moment in time, based on my research, this is how I currently believe these models are generating their output. So, do take it with a grain of salt, and do your own research.
Workforce of the Future
So, while it would be nice to believe that our future work might just be everyone sitting at a desk talking with a AI chat interface all day while they do all the work. This won't be the case for everyone, as AI can and will make mistakes if either the System Prompt or the User Input is not optimally formatted. This isn't a risk I see many businesses wanting to take, a business wants certainty, and chatting directly with these AI bots creates uncertainty, as each worker will have a completely different way of chatting with these AI bots, creating vastly different outputs depending on who asks the AI to do something. This isn't really acceptable from a business point of view, and will only cause a lot of problems and headaches in the future.
Rather, I see the use of System Prompts and the full chat interface being more dedicated to technical workers whom are working on AI projects themselves. End-user everyday workers will instead use a trimmed down interface into these AI systems, where the application would ask the user for various input variables, and then the application written by an expert AI designer whom is themselves trained on creating AI prompts, and testing them, would take in those variables provided by the user, and generate the most appropriate System Prompt and chat conversation with the AI model to perform the required task with the most optimized and predictable prompt information to ensure that the output is always fool-proof. This would require a modern AI application company to really fine-tune the System Prompts and chat questions until they provide very consistent results each time. This may also force the recommendation from these AI application developers to use a specific local model for best results, as each model will always take the same System Prompt and chat questions just a bit differently.
Our world runs upon having a predictable measurement for everything, and so just having everyday workers chat directly with AI models during their everyday job will be vastly unacceptable in the future. Rather, everyday workers will instead use AI through a pre-created application which is purpose created for the task that the everyday worker is performing, and be fine-tuned for that task to lead to a very predictable workflow and output each time.
This does change how a software development job might be in the near future, as more software developers begin integrating more and more AI into their own applications. The modern software developer will also need to be highly trained in LLMs, and how to create System Prompts and chat conversations which will work with the application they are writing to complete the required tasks. One thing I've noticed while working on my own AI applications, is that a task might actually require the application to ask a follow-up question from the previous context to ensure the best results. As an example, you can ask an AI to generate some information you are researching as a large form report of everything you need to know about that subject. Then, a follow-up question would be to summarize all of that information into a single sentence. Follow-up questions can be very helpful depending on the sort of AI application you are working on. This sort of back-and-forth would also be annoying to constantly repeat in a traditional chat interface, as the end-user would need to keep a notepad full of their working System Prompts and chat questions saved somewhere in order to be-able to create similar results for a different topic. This is where AI applications really shine, is that the end-user isn't expected to think much, all they need to know is the input variables, then the AI application already has the perfect fine-tuned System Prompt built right in, along with all the needed chat questions to ensure a consist output each and every time.
Conclusion
Learn how to make amazing System Prompts and how to properly flow chat questions to obtain consistent results. That is if you plan on developing AI applications. If you will just be an everyday user/worker, learn data entry, and how to fill out forms on a website or computer application, as this will be how a lot of everyday workers I feel will eventually be interacting with this technology in the near future. Businesses need consistency in outputs.