OpenAI lets users livestream with ChatGPT.

The AI can now analyze audio and video in real-time and communicate more human-like according to the latest release of OpenAI, GPT Omni. It is a major improvement above ChatGPT. A more conversational and realistic experience is provided by the most recent AI model, GPT-4o. GPT-4 Omni is saw in OpenAI demonstrations helping customers with preparation activities like making calls to customer care to replace an iPhone and making sure they appear presentable for interviews. Additional instances comprise cracking dad jokes, instantaneously translating between two languages, evaluating a game of rock, paper, scissors, and reacting coldly. One demo demonstrates ChatGPT’s initial response upon seeing a user’s puppy.

CEO Sam Altman said in a May 13 blog post, “It feels like AI from the movies, and it’s still a bit surprising to me that it’s real.” Achieving human-level expression and response speeds, he pointed out, signifies a big change. Additionally, according to a recent X post from OpenAI, a version that only accepts text and image inputs launched on May 13. However, the full version is anticipated to arrive in the upcoming weeks. Furthermore, through ChatGPT’s API, GPT-4o will be accessible to both premium and free users of the chat platform. The “o” in GPT-4o, according to OpenAI, stands for “Omni,” emphasizing a shift toward more organic human-computer interactions.

Enhanced Multimodal Capabilities and Performance of OpenAI’s ChatGPT-4o

Compared to OpenAI’s previous AI tools, such as ChatGPT-4, which frequently “loses a lot of information” when pushed to multitask, GPT-4o’s capacity to digest any input of text, audio, and image at the same time is a significant breakthrough.

“GPT-4o is especially better at vision and audio understanding compared to existing models,” according to OpenAI. This includes the ability to detect emotions and breathing patterns in users.

Moreover, OpenAI’s API claims that it is “much faster” and “50% cheaper” than GPT-4 Turbo. Additionally, OpenAI asserts that the new AI tool can react to audio inputs in as short as 2.3 seconds, with an average time of 3.2 seconds. According to the company, this is comparable to human response times in a typical conversation.