Costs of Running LLMs Will 'Drop Significantly' with Nvidia's GH200

Pivot 5: 5 stories. 5 minutes a day. 5 days a week.

1. Costs of Running LLMs Will 'Drop Significantly' with Nvidia's GH200

Nvidia, a dominant force in high-end AI processors, recently introduced the GH200 super chip. This advanced chip is tailored for intricate generative AI tasks, especially LLMs. With the GH200's debut, experts predict a notable decrease in the expenses linked to operating LLMs.

The GH200, while housing the same GPU as Nvidia's premier H100, boasts a memory capacity three times that of its predecessor. This improvement is expected to enhance the performance of intricate AI models, which typically require vast processing power. Before the GH200, many had to spread their models across several GPUs.

Nvidia's stronghold in the generative AI GPU sector is evident. Major cloud services like AWS, Azure, and Google utilize Nvidia's H100 Tensor Core GPUs. Yet, competitors like AMD are on the horizon, planning to ramp up their AI GPU offerings soon.

Read the full story here 

2. Google's Browser-Based Development Gets an AI Boost with Project IDX

Google has launched Project IDX, its foray into AI-enhanced browser-based development environments. IDX supports various frameworks, including Angular, Flutter, and React, and languages such as JavaScript and Dart. More integrations, like Python and Go, are on the horizon.

Leveraging Visual Studio Code — Open Source, Google integrated Codey, its PaLM 2–based foundation model, into IDX. This integration offers intelligent code completion, a ChatGPT/Bard-inspired chatbot for coding queries, and context-aware code actions.

As a cloud-centric IDE, IDX integrates with Google's Firebase Hosting and Google Cloud Functions. It also allows code imports from GitHub repositories. With embedded Android and iOS simulators coming soon, IDX promises a unique full-stack development experience, distinguishing itself from platforms like GitHub’s Copilot and Amazon’s CodeWhisperer.

Read the full story here 

3. ChatGPT Set to Bridge the Gap with Bing Chat Through New Features

Bing Chat has led the AI chatbot scene, but ChatGPT is catching up with upcoming enhancements. Logan Kilpatrick from OpenAI recently shared a list of new ChatGPT features on X (formerly Twitter).

The updates include example prompts, suggested replies, GPT-4 as the default, multi-file uploads for beta users, persistent logins, and keyboard shortcuts. Features like suggested replies and GPT-4 are already in Bing Chat, heightening anticipation for their ChatGPT debut.

These enhancements aim to optimize the ChatGPT experience. Example prompts will guide users, while suggested replies will expedite interactions. Persistent login will also enhance productivity. With the official release imminent, ChatGPT users can expect a more streamlined experience.

Read the full story here

4. Spotify's AI DJ Feature Expands Globally

Spotify is broadening its reach by introducing its AI DJ feature to 50 more countries. Initially available in the US and Canada, this feature provides premium users with algorithmic music suggestions paired with AI-generated DJ commentary. However, the AI DJ, modeled after Spotify’s Xavier Jernigan, communicates solely in English, which might pose challenges in non-English-speaking regions.

The technology behind this feature combines OpenAI's LLM technology and Sonantic's AI voice generation platform, which Spotify acquired last year. This merger offers users AI-generated commentary and insights explaining song choices.

Spotify's commitment to this feature is evident. The company plans to refine and expand the experience, suggesting potential enhancements in the future.

Read the full story here 

5. AI Decodes Keystrokes via Zoom Audio

A recent study reveals an AI's capability to interpret keystrokes using only Zoom audio, achieving a 93% accuracy rate. This discovery, by a UK-based team, highlights the potential risks of audio-based cyberattacks in today's tech landscape.

The AI was trained on a 2021 Macbook Pro, capturing keystrokes via an iPhone microphone and Zoom audio. The results showed 95% and 93% accuracy, respectively. This model surpasses previous keystroke readers in effectiveness.

Ehsan Toreini, a co-author of the study, expressed concerns about the proliferation of microphone-equipped devices. For enhanced security, users can capitalize passwords, vary typing techniques, and use two-step authentication. However, for sensitive discussions on video calls, analog methods might be the safest bet.

Read the full story here 

Listen to Pivot 5 Podcast