Earlier this year, NVIDIA showed the Chat with RTX feature, which is a demonstration of an artificial intelligence service like ChatGPT that uses an open source AI model that can be selected by users at any time. NVIDIA has today rebranded it as NVIDIA ChatRTX, and it's ready for massive testing.
One of the biggest issues shown in the use of large artificial intelligence models such as OpenAI ChatGPT, Microsoft CoPilot, Google Gemini and so on is that confidential data uploaded to such AI services will also be used as machine learning data, and may cause the issue of secret leakage company.
With ChatRTX, users or companies can use a variety of open source AI models for closed artificial intelligence. In fact, with ChatRTX, machine learning will only use the data set set by the user or the company's own policy, and will not use public data that may violate the user's copyright.
Apart from text proms, ChatRTX is also seen to support proms using voice. Depending on the AI model used, ChatRTX currently understands input in English and Mandarin. AI models that can be used include Mistral 7B INT4, Llama 2 7B INT4, CLIP (Contrastive Language Image Pre-training) and Gamma.
Unfortunately for now, NVIDIA also says that it can only understand one prom question at a time, and unlike ChatGPT, Copilot or Gemini, ChatRTX cannot understand additional questions. This function may be introduced in the future when AI models that support TensorRT-LLM are developed in the future.
For now, NVIDIA says that ChatRTX is only supported on computers using NVIDIA GeForce RTX 3000 and 4000 graphics cards that have at least 8GB of VRAM memory and 16GB and above of DDR4 and DDR5 computer memory. In terms of storage, the ChatRTX download file is set at 11GB, and you will need a total of 35GB of storage to install this software.
This software is also only supported on Microsoft Edge and Google Chrome web browsers for now. It is not supported on Mozilla Firefox, but this will be fixed in the future.