Positional encoding is a vital component of Transformer architectures, enabling models to incorporate sequence order into self-attention mechanisms. Rotary Positional Embeddings (RoPE) have become a widely adopted solution due to their compatibility with relative position encoding and computational efficiency. However, RoPE relies on static, input-independent sinusoidal frequency patterns, limiting its ability to model context-sensitive relationships. In this work, we propose CARoPE (Context-Aware Rotary Positional Embedding), a novel generalization of RoPE that dynamically generates head-specific frequency patterns conditioned on token embeddings. This design introduces token- and context-sensitive positional representations while preserving RoPE efficiency and architectural simplicity. CARoPE computes input-dependent phase shifts using a bounded transformation of token embeddings and integrates them into the rotary mechanism across attention heads. We evaluate CARoPE on the FineWeb-Edu-10B dataset using GPT-2 variants trained on next-token prediction tasks. Experimental results show that CARoPE consistently outperforms RoPE and other common positional encoding baselines, achieving significantly lower perplexity, even at longer context lengths. Additionally, CARoPE enables faster training throughput without sacrificing model stability. These findings demonstrate that CARoPE offers a scalable, expressive, and efficient upgrade to existing positional encoding strategies in Transformer models.

Artificial Intelligence (AI) has started playing important role in Healthcare. For Example, an AI system is used to improve early breast cancer detection. Another AI system improves the performance of the Microscope to find cancer cells more efficiently. And, the AI is getting powerful steadily. An AI Algorithm can see and learn to analyze millions of publicly available images on Google Street View to determine the political leanings of a given neighborhood just by looking at the cars on the streets.

Improvements in artificial intelligence hold the potential to put personal health data at risk, a new study shows. Advances in artificial intelligence have created new threats to the privacy of health data, a new UC Berkeley study shows. The study, led by professor Anil Aswani of the Industrial Engineering & Operations Research Department (IEOR) in the College of Engineering and his team, suggests current laws and regulations are nowhere near sufficient to keep an individual's health status private in the face of AI development. The research was released today on JAMA Network Open. In the work, which was funded in part by UC Berkeley's Center for Long-Term Cybersecurity, Aswani shows that by using artificial intelligence, it is possible to identify individuals by learning daily patterns in step data (like that collected by activity trackers, smartwatches and smartphones) and correlating it to demographic data.

Advances in artificial intelligence have created new threats to the privacy of people's health data, a new University of California, Berkeley, study shows. Led by UC Berkeley engineer Anil Aswani, the study suggests current laws and regulations are nowhere near sufficient to keep an individual's health status private in the face of AI development. The research was published Dec. 21 in the JAMA Network Open journal. The findings show that by using artificial intelligence, it is possible to identify individuals by learning daily patterns in step data, such as that collected by activity trackers, smartwatches and smartphones, and correlating it to demographic data. The mining of two years' worth of data covering more than 15,000 Americans led to the conclusion that the privacy standards associated with 1996's HIPAA (Health Insurance Portability and Accountability Act) legislation need to be revisited and reworked.

Advances in artificial intelligence (AI) have created new threats to the privacy of people's health data, a new University of California, Berkeley, a new study shows. Led by University of California Berkeley engineer Anil Aswani, the study suggests current laws and regulations are nowhere near sufficient to keep an individual's health status private in the face of AI development. The research was published Dec. 21 in the JAMA Network Open journal. The findings show that by using artificial intelligence, it is possible to identify individuals by learning daily patterns in step data, such as that collected by activity trackers, smartwatches and smartphones, and correlating it to demographic data. The mining of two years' worth of data covering more than 15,000 Americans led to the conclusion that the privacy standards associated with 1996's HIPAA (Health Insurance Portability and Accountability Act) legislation need to be revisited and reworked.

Advances in artificial intelligence have created new threats to the privacy of people's health data, a new University of California, Berkeley, study shows. Led by UC Berkeley engineer Anil Aswani, the study suggests current laws and regulations are nowhere near sufficient to keep an individual's health status private in the face of AI development. The research was published Dec. 21 in the JAMA Network Open journal. The findings show that by using artificial intelligence, it is possible to identify individuals by learning daily patterns in step data, such as that collected by activity trackers, smartwatches and smartphones, and correlating it to demographic data. The mining of two years' worth of data covering more than 15,000 Americans led to the conclusion that the privacy standards associated with 1996's HIPAA (Health Insurance Portability and Accountability Act) legislation need to be revisited and reworked.

Advances in artificial intelligence (AI) have created new threats to the privacy of health data, a study has found. The study, published in the journal JAMA Network Open, suggests current laws and regulations are nowhere near sufficient to keep an individual's health status private in the face of AI development. The research led by professor Anil Aswani from the University of California -- Berkeley in the US, shows that by using AI, it is possible to identify individuals by learning daily patterns in step data like that collected by activity trackers, smartwatches and smartphones, and correlating it to demographic data. The mining of two years' worth of data covering over 15,000 Americans led to the conclusion that the privacy standards associated with 1996's HIPAA (Health Insurance Portability and Accountability Act) legislation need to be revisited and reworked. "We wanted to use NHANES (the National Health and Nutrition Examination Survey) to look at privacy questions because this data is representative of the diverse population in the US," Aswani said.

Advances in artificial intelligence have created new threats to the privacy of health data, according to a new study by University of California, Berkeley researchers. University of California, Berkeley (UC Berkeley) researchers have found that artificial intelligence (AI) innovations have created new threats to health data privacy against which current laws and regulations cannot adequately safeguard. The researchers demonstrated that AI can be used to identify individuals by learning daily patterns in step data--like that collected by activity trackers, smartwatches, and smartphones--and correlating it to demographic data. Said UC Berkeley's Anil Aswani, "In principle, you could imagine Facebook gathering step data from the app on your smartphone, then buying healthcare data from another company and matching the two. Now they would have healthcare data that's matched to names, and they could either start selling advertising based on that or they could sell the data to others."

January is a peak time for downloading health and fitness apps and putting those Christmas present fitness trackers to work. But do they actually help you stay motivated? After the Christmas self-indulgence comes the inevitable New Year's resolution to get fit, lose weight, and eat more healthily. But while 65% of us make resolutions, only 12% successfully keep to them, polling firm ComRes finds. When Sarah, 34, a law professor from Australia, wanted to lose weight last year, she took the unusual approach of placing bets that she would achieve her exercise goals.