GPT-4o, the ChatGPT macOS software, and conversational AI in voice mode are announced by OpenAI.

According to OpenAI, the GPT-4o is its most sophisticated model, trained end-to-end for text, vision, and audio, meaning that the same neural network processes all inputs and outputs.

GPT-4o, OpenAI’s first artificial intelligence model, can reason across text, images, and audio natively. Since GPT-4o is far more adept than its predecessor at comprehending and interpreting words, visuals, and audios, OpenAI claimed that the “o” in GPT-4o stands for “Omni.” The business also unveiled the ChatGPT software for desktop computers running Apple’s macOS operating system and gave a voice-activated demonstration of conversational AI. Here are the specifics:
GPT-4o

The GPT-4o is described as “a step towards much more natural human-computer interaction” by OpenAI. The organization’s latest iteration of the GPT-4 model can process any combination of text, audio, and image as input and output similar data. The business claimed that the GPT-4o model’s 232 millisecond response time to audio inputs is comparable to a human’s conversational response time.

In terms of English text understanding and coding, the GPT-4o performs similarly to the company’s current GPT-4 Turbo model, which is an iteration of the GPT-4 model. However, it outperforms the GPT-4o in terms of audio understanding. Additionally, the GPT-4o model significantly enhances content in non-English languages.

According to OpenAI, the GPT-4o model significantly enhances visual comprehension. For instance, users can share a photograph of a menu item in many languages with ChatGPT, which is built on GPT-4o, and ask the chatbot to translate it, provide background information on the item, and provide recommendations.

Talk Mode Using GPT-4o

Both paid and free tires in ChatGPT already include the talkback feature in voice mode. OpenAI asserted that the new GPT-4o model, however, significantly enhances it. According to OpenAI, the GPT-4o is its most sophisticated model, trained end-to-end for text, vision, and audio, meaning that the same neural network handles all inputs and outputs. Since all of the data is processed by the same neural network, it effectively reduces latency for a genuine conversational experience and enhances outcomes.

Voice Mode allowed you to communicate with ChatGPT with average latencies of 2.8 seconds (GPT-3.5) and 5.4 seconds (GPT-4) before GPT-4o, according to OpenAI. Three distinct models are used in the data processing pipeline to produce this latency: GPT-3.5 or GPT-4 receives text input and outputs it, a third basic model translates the text back to audio, and one simple model transcribes audio to text. OpenAI claims that a significant amount of data was lost during this procedure to the primary intelligence source, GPT-4.

The Mac app ChatGPT
OpenAI introduced the chatbot app for Apple’s macOS desktops, thereby broadening the ChatGPT software ecosystem. There will be more platform connectivity with the ChatGPT macOS app. Users can ask the chatbot a question by using the keyboard shortcut (Option + Space), which takes them to the ChatGPT conversation page.

OpenAI stated that it is presently developing the Windows version of the application, which is scheduled for release “later this year.”
The ChatGPT macOS app is presently being made available to Plus customers, and in the upcoming weeks, it will also be accessible to users on the free tier.

granting free users access to additional features
Users on the free tier of ChatGPT can access the new GPT-4o model, albeit there is a message restriction. The utilization and demand at the time of use will determine this limit, and ChatGPT will automatically transition to GPT-3.5 when it is achieved. However, a free tier user will now have access to some of the sophisticated features that were previously only available to paying tier members when using chatGPT with GPT-4o.

With GPT-4o, a free tier user can upload documents and images for analysis, summarization, and other purposes. With the new model, ChatGPT’s “Memory” function allows free users to instruct it to remember details for later exchanges. Users on the free tier will now be able to browse and utilize custom bots in the GPT Store. Launched earlier this year for members who pay for access, the GPT store lets users build their own chatbots, or GPTs, and share them with other users. Users on the free tier cannot develop or share bespoke GPTs, but they will have access to the GPT marketplace.

What is still only available to those on subscription tiers

Although features that were previously only available to paid-tier subscribers are now available to free-tier users, the new Voice Mode with GPT-4o will only be available to paid-tier subscribers. ChatGPT Plus members will be able to use the Voice Mode with GPT-4o model compatibility in the upcoming weeks, while Team and corporate users will have access to it sooner. Additionally, OpenAI is making the GPT-4o model available to paying customers with “fewer limitations.”
Users of ChatGPT Plus and Teams are already able to access the new model, and Enterprise users will be able to do the same in the near future. According to the corporation, the message limit for Plus customers will be up to five times higher than that of free users, and the limit will be much higher for Team and Enterprise users.