In 1916, hybrid cars could've changed history. But Ford wouldn't allow it. Henry Ford's monopoly on the automobile industry meant that hybrids wouldn't see the light of day for decades. In 1916, Clinton Edgar Woods, a forgotten automobile inventor, designed the first commercial hybrid cars. But Ford's Model T had already cornered the market.

Alignment serves as an important step to steer large language models (LLMs) towards human preferences. In this paper, we explore contrastive post-training techniques for alignment by automatically constructing preference pairs from multiple models of varying strengths (e.g., InstructGPT, ChatGPT and GPT-4). We carefully compare the contrastive techniques of SLiC and DPO to SFT baselines and find that DPO provides a step-function improvement even after continueing SFT saturates. We also explore a data curriculum learning scheme for contrastive posttraining, which starts by learning from "easier" pairs and transitioning to "harder" ones, which further improves alignment. Finally, we scale up our experiments to train with more data and larger models like Orca. Remarkably, contrastive post-training further improves the performance of Orca, already a state-of-the-art instruction learning model tuned with GPT-4 outputs, to exceed that of ChatGPT. The rapid evolution of Large Language Models (LLMs) has ushered in a new era of natural language processing capabilities. These models, when scaled to billions of parameters and pretrained over trillions of text tokens, demonstrate unprecedented proficiency in a wide array of tasks (Brown et al., 2020; Chowdhery et al., 2022). Various post-training procedures like supervised instruction tuning and Reinforcement Learning from Human Feedback (RLHF) fine-tune pretrained LLMs to better align with human expectations and preferences (Ouyang et al., 2022; OpenAI, 2023; Touvron et al., 2023a). This additional alignment procedure is crucial, because the pretraining objective of essentially predicting the next token in a text sequence is known to produce LLMs whose outputs are at times incorrect, irrelevant, or unsafe (Bai et al., 2022a). Traditionally, these post-training techniques rely on human preference annotations to inform an LLM which behaviors it ought to adopt in the scenario at hand. For instance, RLHF fits a reward model on these preference pairs, against which a LLM policy is then optimized (Ziegler et al., 2019; Bai et al., 2022a; Touvron et al., 2023b). However, such human feedback is expensive to obtain and often noisy (Stiennon et al., 2020; Ouyang et al., 2022; Bai et al., 2022a). To align an LLM without human feedback, other methods such as Reinforcement Learning from AI Feedback (RLAIF) harvest preference signals via automatic feedback from another LLM (Lee et al., 2023; Bai et al., 2022b). However, studies have found AI feedback has a low agreement rate with humans (Perez et al., 2022; Casper et al., 2023b; Lee et al., 2021). Also, these methods suffer from the same drawbacks as RLHF, such as reward hacking (Skalse et al., 2022).

Which three-word phrase should always be spoken cautiously? All of them, actually, but that last one -- depending on your choice of ride, a full tank of gas can now cost you within fumes-sniffing distance of a hundred bucks. How did it come to this -- again? Los Angeles is a complex place. In this weekly feature, Patt Morrison is explaining how it works, its history and its culture.

When GM's very first car, the Chevrolet Classic 6, appeared on Detroit's streets in 1912, it ran on gasoline. Some 112 years later, in 2034, the very last GM car that runs on gasoline is scheduled to roll off the assembly line. Starting in 2035, GM intends to make only electric cars, from its least expensive model, the $4,000 Hongguang Mini in China, to the "handcrafted" Cadillac Celestiq at $200,000-plus. The other major car companies, from Ford and Toyota to Volkswagen and Volvo, are heading in the same direction. Governments are also driving the shift.

IIT Hyderabad Researchers are using computational methods to understand the factors and impediments in incorporating biofuels into the fuel sector in India. This work has been spurred by the increasing need to replace fossil fuels by bio-derived fuels, which, in turn, is driven by the dwindling fossil fuel reserves all over the world, and pollution issues associated with the use of fossil fuels. The model developed by the IIT Hyderabad team has shown that in the area of bioethanol integration into mainstream fuel use, the production cost is the highest (43 per cent) followed by import (25 per cent), transport (17 per cent), infrastructure (15 per cent) and inventory (0.43 per cent) costs. The model has also shown that feed availability to the tune of at least 40 per cent of the capacity is needed to meet the projected demands. A unique feature of this work is that the framework considers revenue generation not only as an outcome of sales of the biofuel but also in terms of carbon credits via greenhouse gas emission savings throughout the project lifecycle.

There are many never-ending debates between Republicans and Democrats. Impeach vs. don't impeach; capital punishment vs. life in prison; wall vs. no wall; legalizing marijuana vs. not; self-driving cars vs. human drivers; Red Sox vs. Yankees; takeout vs. home-cooked; or Gone With the Wind vs. any other movie. All of these issues are stunningly important, right up to the second where cataclysm falls and creates a nightmare scenario that so many fear. That cataclysm is a complete loss of electricity and every mode of convenience and survival we take for granted. IS NORTH KOREA'S EMP THREAT REAL OR'SOMETHING OUT OF A JAMES BOND MOVIE'?

A lit cigarette burns at around 600-degrees Celsius (1,100 F). The auto-ignition point for gasoline is less than half that. Somehow, despite these facts, people smoke while pumping gas. In fact, it's such a prevalent problem that Microsoft developed an AI-powered alarm system to help gas station employees crack down on offenders. It's difficult to imagine the amount of idiotic hubris it takes to disregard basic thermodynamics, but some people simply won't be bothered by certain doom.

Range anxiety, the bugaboo of all-electric driving, is even more frightening for all-electric flying, where running out of power has worse consequences than having to pull over to the side of the road. A solution now comes from Workhorse, an Ohio-based firm. It has a passenger-carrying air taxi, called the SureFly, which combines the company's expertise in partially automated operation, from its drone business, and in hybrid-electric propulsion, from its truck business. The craft's eight counter-rotating motors each drive a carbon-fiber rotor, and the power comes from a generator cranked by an internal-combustion engine. You can fly 110 kilometers (70 miles) on a tank, then refill in minutes.

Honda's revamped Clarity fuel cell sedan is a technological marvel, but hydrogen fuel stations remain scarce. The dogged persistence Honda has shown in its decades-long quest to perfect hydrogen as a zero-emission replacement for gasoline can be seen as quixotic or futile. But the quirky company that makes motorcycles, lawnmowers, jets, humanoid robots, boat engines and a few million cars soldiers on, along with GM, Toyota and Hyundai, in the face of haphazard government support for hydrogen, minimal consumer awareness and withering critics like Elon Musk, a tireless advocate for Tesla's battery-powered cars. All the sweat equity Honda engineers have invested in its fuel cell program, year after year, has yielded a remarkable new version of the Clarity sedan, the most compelling argument yet of the potential of hydrogen cars still hold. Yet for all its technological sophistication, Clarity's fate remains to a skinny network of California hydrogen stations that's expanding slowly, with new headwinds from a Trump Administration that's shown no willingness to aid carbon-cutting technologies.

If an object has a battery in it or a plug at the end of it, it won't be long before that item is intelligent – although Kevin Kelly, the founder editor of WIRED, questions whether intelligence is really the word we want to be using. "It's best I think to think of these intelligences as smartness instead of intelligence, because we have a lot of baggage with the idea of intelligence," says Kelly. He suggests a new verb: cognify, and also cognification. This is what Kelly terms the second industrial revolution. The first saw us put the power of muscle into objects in the form of energy – steam, gasoline, electricity – literally giving things like cars a certain amount of'horsepower'.