In this paper, we study the role of initialization in Low Rank Adaptation (LoRA) as originally introduced in Hu et al. (2021). Essentially, to start from the pretrained model as initialization for finetuning, one can either initialize B to zero and A to random (default initialization in PEFT package), or vice-versa. In both cases, the product BA is equal to zero at initialization, which makes finetuning starts from the pretrained model. These two initialization schemes are seemingly similar. They should in-principle yield the same performance and share the same optimal learning rate. We demonstrate that this is an incorrect intuition and that the first scheme (initializing B to zero and A to random) on average yields better performance compared to the other scheme. Our theoretical analysis shows that the reason behind this might be that the first initialization allows the use of larger learning rates (without causing output instability) compared to the second initialization, resulting in more efficient learning of the first scheme. We validate our results with extensive experiments on LLMs.

Musk originally filed his lawsuit at the beginning of March, arguing that OpenAI had breached its commitment to early investors and the public to build AI for the benefit of humanity when it began making money. At the time, OpenAI executives blamed Musk's lawsuit on his not being a part of the company as it was seeing massive success. Musk did not immediately respond to a request for comment Tuesday. A spokesperson for OpenAI declined to comment.

Elon Musk dropped a lawsuit alleging OpenAI and its chief executive officer Sam Altman breached a founding promise last year by prioritizing profits over humanity. The billionaire withdrew his complaint a day before a California judge was set to hear OpenAI's request for dismissal. Musk had accused the company of becoming a "de facto subsidiary" of Microsoft Corp. in violation of a founding agreement to be a non-profit that developed artificial intelligence "for the benefit of humanity." OpenAI and Musk have been engaged in a well-publicized battle since well before the court case. Musk was an early backer of the startup and part of its founding team, before he had a falling out with the company.

Elon Musk has moved to dismiss his lawsuit accusing ChatGPT maker OpenAI and its CEO Sam Altman of abandoning the startup's original mission of developing artificial intelligence for the benefit of humanity. Musk launched the suit against Altman in February, and the case had been slowly working its way through the California court system. There was no indication until Tuesday that Musk planned to drop the suit; only a month ago, his lawyers filed a challenge that forced the judge hearing the case to remove himself. Musk's request for a dismissal contained no reason behind the decision. A San Francisco superior court judge was scheduled on Wednesday to hear Altman and OpenAI's argument for throwing the case out.

Elon Musk dropped a lawsuit against OpenAI one day before a judge in California state court was set to hear OpenAI's request for dismissal. Musk's suit, which was filed in February, had accused OpenAI co-founders Sam Altman and Greg Brockman of violating the company's non-profit status and instead prioritizing profits over using AI to help humanity. In the 35-page suit, Musk had alleged that OpenAI had become a "closed-source de facto subsidiary" of Microsoft, which invested 13 billion in the company and owns a 49 percent stake. Microsoft uses OpenAI's technology to power Copilot, the company's generative AI tools that are deeply integrated in products like Windows and Office. OpenAI had reportedly requested for the lawsuit to be dismissed, arguing that Musk would use any information that emerged as a result to get access to the company's "proprietary records and technology."

Yesterday, following Apple's announcement of a partnership with OpenAI to integrate support for ChatGPT into the company's devices, Elon Musk did what he always does: he tweeted. The owner of X wrote, on X, that he would ban Apple devices at his companies "If Apple integrates OpenAI at the OS level.". And to that I say: Go right ahead. And while you're at it, remove your company's software from Apple's App Store too. Musk's companies (at least the major ones) currently include Tesla, SpaceX, X, X AI and Neuralink.

One of the reasons for consumers to buy into Microsoft's Copilot Pro, Custom GPT Builder, is quietly being killed off. Microsoft said that GPT Builder is being retired, and if you have bought into it, you'll have to remove all of your data by July 14. Unfortunately, Microsoft doesn't consider the ability to build GPT models a consumer priority, the company wrote in a support document. Microsoft Copilot is a "neutral," Microsoft-specific version of Copilot. You can think of GPT Builder as a way to personalize Copilot in much the same way that you can personalize your PC: GPT Builder allowed you to train a custom Copilot on your own data, with your own style.

Elon Musk took aim at Apple after it announced a partnership to use OpenAI's technology on its devices. Musk took to his social media platform X (formerly Twitter) to voice concerns about Apple potentially integrating OpenAI at the operating system level, after the tie-up was announced Monday, calling the deal a security risk. "If Apple integrates OpenAI at the OS level, then Apple devices will be banned at my companies," Musk wrote. "Visitors will have to check their Apple devices at the door, where they will be stored in a Faraday cage." Apple CEO Tim Cook announced the company's new AI offering, which the company dubs "Apple Intelligence," during the tech giant's developers conference on June 10.

Apple's annual developer shindig kicked off with its traditional keynote outlining all the new tricks its products will soon do. There are big changes for iOS 18, iPadOS 18, macOS Sequoia and watchOS 11, not to mention visionOS 2. Some highlights include a standalone Passwords app, better health metrics on the Watch and Apple Intelligence, its own spin on AI. There's more to learn about, so keep reading to learn all the biggest stories from the show. Apple may integrate Google's Gemini AI into iOS in the future You can get these reports delivered daily direct to your inbox. Apple has finally bowed to pressure, bringing AI to its devices in the form of Apple Intelligence, powered by OpenAI.

Elon Musk announced Monday that he'll ban Apple devices from his companies' premises if the iPhone creator goes forward with its planned OpenAI integration. Apple held its annual Worldwide Developer's Conference Monday, and the biggest takeaway was OpenAI's ChatGPT will be paired with Apple's assistant Siri. In one example shown, Siri recommended that the iPhone user consult ChatGPT for further dinner recipe ideas, flagging all the way that these new answers were coming from OpenAI's chatbot and advising users to'check important info for mistakes'. Musk responded to the partnership on X, saying employees and visitors to his companies would'have to check their Apple devices at the door' where they will be stored in a'Faraday cage' – a cage that blocks electromagnetic fields. He continued: 'If Apple integrates OpenAI at the OS level, then Apple devices will be banned at my companies.