Goto

Collaborating Authors

 Rochester


CTIBench: A Benchmark for Evaluating LLMs in Cyber Threat Intelligence Dipkamal Bhusal Rochester Institute of Technology Rochester Institute of Technology Rochester, NY, USA

Neural Information Processing Systems

Cyber threat intelligence (CTI) is crucial in today's cybersecurity landscape, providing essential insights to understand and mitigate the ever-evolving cyber threats. The recent rise of Large Language Models (LLMs) have shown potential in this domain, but concerns about their reliability, accuracy, and hallucinations persist. While existing benchmarks provide general evaluations of LLMs, there are no benchmarks that address the practical and applied aspects of CTI-specific tasks. To bridge this gap, we introduce CTIBench, a benchmark designed to assess LLMs' performance in CTI applications. CTIBench includes multiple datasets focused on evaluating knowledge acquired by LLMs in the cyber-threat landscape. Our evaluation of several state-of-the-art models on these tasks provides insights into their strengths and weaknesses in CTI contexts, contributing to a better understanding of LLM capabilities in CTI.


New Capability to Look Up an ASL Sign from a Video Example

arXiv.org Artificial Intelligence

Looking up an unknown sign in an ASL dictionary can be difficult. Most ASL dictionaries are organized based on English glosses, despite the fact that (1) there is no convention for assigning English-based glosses to ASL signs; and (2) there is no 1-1 correspondence between ASL signs and English words. Furthermore, what if the user does not know either the meaning of the target sign or its possible English translation(s)? Some ASL dictionaries enable searching through specification of articulatory properties, such as handshapes, locations, movement properties, etc. However, this is a cumbersome process and does not always result in successful lookup. Here we describe a new system, publicly shared on the Web, to enable lookup of a video of an ASL sign (e.g., a webcam recording or a clip from a continuous signing video). The user submits a video for analysis and is presented with the five most likely sign matches, in decreasing order of likelihood, so that the user can confirm the selection and then be taken to our ASLLRP Sign Bank entry for that sign. Furthermore, this video lookup is also integrated into our newest version of SignStream(R) software to facilitate linguistic annotation of ASL video data, enabling the user to directly look up a sign in the video being annotated, and, upon confirmation of the match, to directly enter into the annotation the gloss and features of that sign, greatly increasing the efficiency and consistency of linguistic annotations of ASL video data.


Censoring chemical data to mitigate dual use risk

arXiv.org Artificial Intelligence

The dual use of machine learning applications, where models can be used for both beneficial and malicious purposes, presents a significant challenge. This has recently become a particular concern in chemistry, where chemical datasets containing sensitive labels (e.g. toxicological information) could be used to develop predictive models that identify novel toxins or chemical warfare agents. To mitigate dual use risks, we propose a model-agnostic method of selectively noising datasets while preserving the utility of the data for training deep neural networks in a beneficial region. We evaluate the effectiveness of the proposed method across least squares, a multilayer perceptron, and a graph neural network. Our findings show selectively noised datasets can induce model variance and bias in predictions for sensitive labels with control, suggesting the safe sharing of datasets containing sensitive information is feasible. We also find omitting sensitive data often increases model variance sufficiently to mitigate dual use. This work is proposed as a foundation for future research on enabling more secure and collaborative data sharing practices and safer machine learning applications in chemistry.


ChatGPT launches boom in AI-written e-books on Amazon

The Japan Times

SAN FRANCISCO โ€“ Until recently, Brett Schickler never imagined he could be a published author, though he had dreamed about it. But after learning about the ChatGPT artificial intelligence program, Schickler figured an opportunity had landed in his lap. "The idea of writing a book finally seemed possible," said Schickler, a salesman in Rochester, New York. "I thought'I can do this.'" Using the AI software, which can generate blocks of text from simple prompts, Schickler created a 30-page illustrated children's e-book in a matter of hours, offering it for sale in January through Amazon.com


The Kindle Store has a prolific new author: ChatGPT

Engadget

ChatGPT is listed as the author or co-author of at least 200 books on Amazon's Kindle Store, according to Reuters. However, the actual number of bot-written books is likely much higher than that since Amazon's policies don't explicitly require authors to disclose their use of AI. "I could see people making a whole career out of this," said Brett Schickler, a Rochester, NY salesman who published a children's book on the Kindle Store. "The idea of writing a book finally seemed possible." Schickler's self-published story, The Wise Little Squirrel: A Tale of Saving and Investing, is a 30-page children's story -- written and illustrated by AI -- selling for $2.99 for a digital copy and $9.99 for a printed version. Although Schickler says the book has earned him less than $100 since its January release, he only spent a few hours creating it with ChatGPT prompts like "write a story about a dad teaching his son about financial literacy."


Jim Rock steps down as Seegrid CEO

#artificialintelligence

Jim Rock has stepped down as CEO of Seegrid after nearly a decade. Seegrid is a Pittsburgh-based developer of autonomous mobile robots (AMRs) for logistics operations. Joe Pajer has been named the new CEO of Seegrid, effective immediately. Pajer, who obtained both his master's in management and bachelor's in civil engineering, engineering and public policy from Carnegie Mellon University, most recently served as the president and CEO of Calero Software, a Rochester, New York-based provider of communications and cloud lifecycle management solutions that merged with a competitor โ€“ MDSL โ€“ in 2019. He served as president and CEO of two other companies: Thinklogical, a Connecticut-based IT firm, and Vocollect, a Pittsburgh-based software maker that now operates as a division of Honeywell Safety & Productivity Solutions.


L3Harris to developing artificial intelligence systems for DOD โ€“ IAM Network

#artificialintelligence

L3Harris will research, develop and demonstrate the interface using data science techniques under a new multimillion-dollar contract to support DOD applications. L3Harris will perform the work in Rochester, N.Y., Melbourne, Fla., and Herndon, Va.


Large-scale nonlinear Granger causality: A data-driven, multivariate approach to recovering directed networks from short time-series data

arXiv.org Machine Learning

To gain insight into complex systems it is a key challenge to infer nonlinear causal directional relations from observational time-series data. Specifically, estimating causal relationships between interacting components in large systems with only short recordings over few temporal observations remains an important, yet unresolved problem. Here, we introduce a large-scale Nonlinear Granger Causality (lsNGC) approach for inferring directional, nonlinear, multivariate causal interactions between system components from short high-dimensional time-series recordings. By modeling interactions with nonlinear state-space transformations from limited observational data, lsNGC identifies casual relations with no explicit a priori assumptions on functional interdependence between component time-series in a computationally efficient manner. Additionally, our method provides a mathematical formulation revealing statistical significance of inferred causal relations. We extensively study the ability of lsNGC to recovering network structure from two-node to thirty-four node chaotic time-series systems. Our results suggest that lsNGC captures meaningful interactions from limited observational data, where it performs favorably when compared to traditionally used methods. Finally, we demonstrate the applicability of lsNGC to estimating causality in large, real-world systems by inferring directional nonlinear, multivariate causal relationships among a large number of relatively short time-series acquired from functional Magnetic Resonance Imaging (fMRI) data of the human brain.


Learning Simple Thresholded Features with Sparse Support Recovery

arXiv.org Machine Learning

The thresholded feature has recently emerged as an extremely efficient, yet rough empirical approximation, of the time-consuming sparse coding inference process. Such an approximation has not yet been rigorously examined, and standard dictionaries often lead to non-optimal performance when used for computing thresholded features. In this paper, we first present two theoretical recovery guarantees for the thresholded feature to exactly recover the nonzero support of the sparse code. Motivated by them, we then formulate the Dictionary Learning for Thresholded Features (DLTF) model, which learns an optimized dictionary for applying the thresholded feature. In particular, for the $(k, 2)$ norm involved, a novel proximal operator with log-linear time complexity $O(m\log m)$ is derived. We evaluate the performance of DLTF on a vast range of synthetic and real-data tasks, where DLTF demonstrates remarkable efficiency, effectiveness and robustness in all experiments. In addition, we briefly discuss the potential link between DLTF and deep learning building blocks.