Following last week’s launch of Gemini 1.0 Ultra under the Bard rebrand, Google introduces its latest model, Gemini 1.5 Pro, aimed at rivaling GPT-4. This successor to Gemini 1.0 Pro, which currently drives the free iteration of Gemini (formerly Bard), marks another stride in Google’s AI advancements.
While the Gemini 1.0 models boast a context window of up to 32K tokens, the 1.5 Pro model extends the standard context length to 128K tokens. Moreover, it supports an extensive context window of up to 1 million tokens, surpassing both GPT-4 Turbo’s 128K and Claude 2.1’s 200K tokens.
Gemini 1.5 Pro utilizes a Mixture-of-Experts (MoE) architecture.
According to Google, Gemini 1.5 Pro is classified as a mid-size model, yet it achieves nearly identical performance to Gemini 1.0 Ultra while requiring less computational power. This feat is attributed to the use of the Mixture-of-Experts (MoE) architecture, akin to OpenAI’s GPT-4 model. It marks Google’s debut of an MoE model, departing from the conventional single dense model approach.
The MoE architecture comprises multiple smaller expert models that activate based on the specific task being performed. By employing specialized models for distinct tasks, it enhances efficiency and yields superior results.
Regarding the expansive context window of Gemini 1.5 Pro, it possesses the capability to handle substantial volumes of data in a single operation. According to Google, the 1 million context length enables processing of 700,000 words, equivalent to 1 hour of video, 11 hours of audio, or codebases comprising over 30,000 lines of code.
To assess the retrieval prowess of Gemini 1.5 Pro, given its extensive context window, Google conducted the Needle In A Haystack challenge. According to the company, the model successfully recalled the needle (text statement) 99% of the time.
In our evaluation comparing Gemini 1.0 Ultra with GPT-4, we conducted a similar test. However, Gemini 1.0 Ultra was unable to retrieve the statement. We plan to conduct thorough testing of the new Gemini 1.5 Pro model and will share the outcomes accordingly.
Just to clarify, the 1.5 Pro model is currently in preview, accessible only to developers and customers for testing through AI Studio and Vertex AI. You can click the link to join the waitlist. Access to the model will be free during the testing phase.
For the reason that the admin of this site is working, no uncertainty very quickly it will be renowned, due to its quality contents.