Who among our readers thought that a physical AI device is your next everyday device instead of a smartphone with AI? It seems that the idea of a physical AI device continues to become more and more irrelevant now that Anthropic has come up with a new model that can control a user's computer.
According to Anthropic, their new model is the next generation algorithm for self-educating AI. The new Claude 3.5 Sonnet model has been updated with the Computer Use API. With this new model it can mimic the keys pressed, the mouse moved and learn the user's computer usage style to become a virtual personal assistant that is more customized to the user himself.
This model basically counts the pixels of the user's movement, then takes a screenshot of the screen to understand the content before it processes the user's prom to perform the next task. The example shown is completing a Microsoft Excel document by finding information on the web and filling it in the document. However, in Anthropic's own tests, this model can only mimic about half of user activity and is not yet ready to replace human work.
This new model is also known as Large Action Model by some other AI companies. Anthropic uses the same term as Honor which is AI Agent. For now, it's limited to developer testing through the Amazon Bedrock and Google Cloud Vertex AI platforms. For regular users, the more powerful Claude 3.5 Sonnet model without the Computer Use API can now be used.