The Cancer Registry of Norway (CRN) uses an automated cancer registration support system (CaReSS) to support core cancer registry activities, i.e, data capture, data curation, and producing data products and statistics for various stakeholders. GURI is a core component of CaReSS, which is responsible for validating incoming data with medical rules. Such medical rules are manually implemented by medical experts based on medical standards, regulations, and research. Since large language models (LLMs) have been trained on a large amount of public information, including these documents, they can be employed to generate tests for GURI. Thus, we propose an LLM-based test generation and differential testing approach (LLMeDiff) to test GURI. We experimented with four different LLMs, two medical rule engine implementations, and 58 real medical rules to investigate the hallucination, success, time efficiency, and robustness of the LLMs to generate tests, and these tests' ability to find potential issues in GURI. Our results showed that GPT-3.5 hallucinates the least, is the most successful, and is generally the most robust; however, it has the worst time efficiency. Our differential testing revealed 22 medical rules where implementation inconsistencies were discovered (e.g., regarding handling rule versions). Finally, we provide insights for practitioners and researchers based on the results.

A few hours after dark one evening earlier this month, a small quadcopter drone lifted off from the parking lot of Ben-Gurion University in Beersheba, Israel. It soon trained its built-in camera on its target, a desktop computer's tiny blinking light inside a third-floor office nearby. The pinpoint flickers, emitting from the LED hard drive indicator that lights up intermittently on practically every modern Windows machine, would hardly arouse the suspicions of anyone working in the office after hours. But in fact, that LED was silently winking out an optical stream of the computer's secrets to the camera floating outside. That data-stealing drone, shown in the video below, works as a Mr. Robot-style demonstration of a very real espionage technique.