Are you familiar with Qlik and the OpenAI API? If not, let me tell you about my experience integrating them together. I used the REST API connector in the Qlik system to connect to the OpenAI engine and would like to share some insights from my tests. It’s important to note that there are two models to distinguish: GPT-3 (with API) and ChatGPT (no API available yet). The former is a deep learning neural network model with over 175 billion machine learning parameters, while the latter is a large language model developed based on the GPT-3.5 language model with several hundred million parameters.
Here are some steps to get started:
- Download my sample app with script.
- Choose which tables from your data model and what range of data you want to send to GPT-3. Prepare ONE TABLE (as recommended by GPT-3) as the source of data for analysis. You can send two or more tables in the appropriate JSON format, but the recommendation is to use one table as a source for analysis. Hide or map sensitive data.
- Prepare prompts (text hints and requests) for the OpenAI engine.
- Send source data and prompts using the Qlik REST Connector. You can send only text prompts without any data, but then you will get a general result not related to your data but suitable for presentation and further analysis.
- Get a table with the result and token usage statistics. You will find the answer in one table cell. Map sensitive data back and override the GPT-3 result. Show on the dashboard.
If you don’t want to send company data but only general questions, follow these steps instead:
- Prepare prompts (text hints and requests) for the OpenAI engine. Experiment with the “temperature” parameter between 0 and 1, where 0 is the most predictable and 1 is the most random.
- Send prompts using the Qlik REST Connector.
- Get a table with the result and token usage statistics. You will find the answer in one table cell. Show on the dashboard.
It’s important to note that you can’t query the OpenAI API a second time for the same data package you sent in the previous query. You have to fit everything into one query every time.
Before you get started, there are some prerequisites to follow:
- Go to the “Completions” section of the OpenAI API documentation for information on how to use the API.
- Create an account and start with the Playground to chat with GPT-3 and ask about your data columns, best format, etc. Check the different model settings.
- Create your Secret API Key for Qlik REST Connector authorization.
- Follow token usage. Tokens are calculated based on what you send (prompt) and what you get (completion). Depending on the model used, requests can use up to 4097 tokens (about 3000 words) shared between prompt and completion. If your prompt is 4000 tokens, your completion can be 97 tokens at most.
- Keep pricing in mind. If you’re a new player, you have a few months and $18 to spend.
- Be aware of data security. The OpenAI API theoretically does not save what you send, but it’s best to be safe. Anonymize your data package by not sending client names, personal information or exact addresses, email addresses, phone numbers, and replacing names with identifiers. Aggregate data and use a mapping table for sensitive data as a bridge between your model and the OpenAI analysis package.
We can now display the results in our Qlik app using tables, charts, or any other visualizations that we choose.
👍 Conclusion
In this article, we have shown how to use the Qlik REST Connector to connect to the OpenAI API and integrate GPT-3 into our Qlik app. We have also provided some tips on how to prepare our data and prompts for the OpenAI engine and how to map sensitive data back to its original form.
By integrating GPT-3 into our Qlik app, we can take advantage of its powerful natural language processing capabilities to analyze and understand our data in new ways. This can help us to uncover insights and patterns that we might.