Document recommendation systems for locating relevant literature have mostly relied on methods developed a decade ago. This is largely due to the lack of a large offline gold-standard benchmark of relevant documents that cover a variety of research fields such that newly developed literature search techniques can be compared, improved and translated into practice. To overcome this bottleneck, we have established the RElevant LIterature SearcH consortium consisting of more than 1500 scientists from 84 countries, who have collectively annotated the relevance of over 180 000 PubMed-listed articles with regard to their respective seed (input) article/s. The majority of annotations were contributed by highly experienced, original authors of the seed articles. The collected data cover 76% of all unique PubMed Medical Subject Headings descriptors. No systematic biases were observed across different experience levels, research fields or time spent on annotations.
E.A. Feigenbaum and J. Feldman (Eds.). Computers and Thought. McGraw-Hill, 1963. This collection includes twenty classic papers by such pioneers as A. M. Turing and Marvin Minsky who were behind the pivotal advances in artificially simulating human thought processes with computers. All Parts are available as downloadable pdf files; most individual chapters are also available separately. COMPUTING MACHINERY AND INTELLIGENCE. A. M. Turing. CHESS-PLAYING PROGRAMS AND THE PROBLEM OF COMPLEXITY. Allen Newell, J.C. Shaw and H.A. Simon. SOME STUDIES IN MACHINE LEARNING USING THE GAME OF CHECKERS. A. L. Samuel. EMPIRICAL EXPLORATIONS WITH THE LOGIC THEORY MACHINE: A CASE STUDY IN HEURISTICS. Allen Newell J.C. Shaw and H.A. Simon. REALIZATION OF A GEOMETRY-THEOREM PROVING MACHINE. H. Gelernter. EMPIRICAL EXPLORATIONS OF THE GEOMETRY-THEOREM PROVING MACHINE. H. Gelernter, J.R. Hansen, and D. W. Loveland. SUMMARY OF A HEURISTIC LINE BALANCING PROCEDURE. Fred M. Tonge. A HEURISTIC PROGRAM THAT SOLVES SYMBOLIC INTEGRATION PROBLEMS IN FRESHMAN CALCULUS. James R. Slagle. BASEBALL: AN AUTOMATIC QUESTION ANSWERER. Green, Bert F. Jr., Alice K. Wolf, Carol Chomsky, and Kenneth Laughery. INFERENTIAL MEMORY AS THE BASIS OF MACHINES WHICH UNDERSTAND NATURAL LANGUAGE. Robert K. Lindsay. PATTERN RECOGNITION BY MACHINE. Oliver G. Selfridge and Ulric Neisser. A PATTERN-RECOGNITION PROGRAM THAT GENERATES, EVALUATES, AND ADJUSTS ITS OWN OPERATORS. Leonard Uhr and Charles Vossler. GPS, A PROGRAM THAT SIMULATES HUMAN THOUGHT. Allen Newell and H.A. Simon. THE SIMULATION OF VERBAL LEARNING BEHAVIOR. Edward A. Feigenbaum. PROGRAMMING A MODEL OF HUMAN CONCEPT FORMULATION. Earl B. Hunt and Carl I. Hovland. SIMULATION OF BEHAVIOR IN THE BINARY CHOICE EXPERIMENT Julian Feldman. A MODEL OF THE TRUST INVESTMENT PROCESS. Geoffrey P. E. Clarkson. A COMPUTER MODEL OF ELEMENTARY SOCIAL BEHAVIOR. John T. Gullahorn and Jeanne E. Gullahorn. TOWARD INTELLIGENT MACHINES. Paul Armer. STEPS TOWARD ARTIFICIAL INTELLIGENCE. Marvin Minsky. A SELECTED DESCRIPTOR-INDEXED BIBLIOGRAPHY TO THE LITERATURE ON ARTIFICIAL INTELLIGENCE. Marvin Minsky.
> Science's COVID-19 coverage is supported by the Pulitzer Center. In a normal summer, Appledore Island, a 39-hectare outcrop 12 kilometers off the coast of Maine and New Hampshire, becomes a classroom. Students from high school to graduate level live in close quarters, eat in a communal dining hall, and work shoulder to shoulder to explore the biology of the shore and waters in 18 courses organized by the Shoals Marine Laboratory. But this summer, with the pandemic surging, students have stayed home. Instead, a skeleton staff on Appledore is streaming field trips and dissections of fish and invertebrates and setting up cameras to gather data for students. Rather than leading students around the island, coastal restoration ecologist Gregg Moore from the University of New Hampshire (UNH), Durham, hauls a backpack full of equipment: “a dual modem with two different cellular carriers, a signal-boosting directional antenna, and a large DC power source,” he says. The equipment allows him to teach 12 remote students—twice the course's usual enrollment—basic techniques of coastal ecology. Moore's is just one of hundreds of lab and field courses forced online by COVID-19—“a seismic shift for those who were not already involved in distance or online education,” says Martin Storksdieck, a science education researcher at Oregon State University, Corvallis. Some researchers worry students will miss out on certain practical and problem-solving skills and won't be able to judge whether the hands-on work of a scientist is a good fit for them. But instructors are developing high-tech ways to simulate the field and lab experiences. “I would say [these courses] are not virtual,” says Jennifer Seavey, director of the Shoals lab. “They are real.” And some advantages are emerging. By lowering geographical and financial barriers, Seavey says, “Virtual field courses are democratizing fieldwork.” The shift has taken ingenuity. “Professors must get creative and use a combination of what is available,” including online videos and free or commercially available online labs, says Mildred Pointer, a physiologist at Howard University who is working on a fall course in general biology. No single tool meets all their needs, Pointer says. As the pandemic gained momentum, emails flew among the leaders of the National Association of Geoscience Teachers. Many U.S. geology majors must take a “capstone” field course to graduate. The cancellation of more than three-quarters of these courses jeopardized graduation for many majors. So the association invited instructors to develop learning objectives that did not depend on students doing fieldwork. It also compiled online exercises to help the 29 field courses that have moved online this summer. Lessons range from “Orienteering in Minecraft” to “Geology of Yosemite Valley,” which includes a 43-stop Google Earth tour with photos and embedded text. Like Moore, geoscientist Jim Handschy wanted to give remote students “as close to the real experience as possible.” He runs Indiana University's Judson Mead Geologic Field Station in Montana, which had enrolled 60 students before classes were canceled in March. He and a few instructors visited each outcrop in their course plan, filmed the rocks and landscape, and captured magnified views of samples. Each week, the class delves deeper into the rock layers and their history. For their final project, students digitally map a 3100-hectare landscape. Shannon Dulin, a geologist at the University of Oklahoma, Norman, who just finished teaching a field course, sees the value of learning how to survey a landscape without setting foot on it. On their class evaluations, her students said they gained unexpected skills. “And these are skills they are going to need on the job,” she adds, as geologists are increasingly being asked to evaluate sites they don't visit. In other fields, hands-on learning takes place in labs. Typically, students work in pairs and share equipment, “so there are a lot of issues about virus transmission,” says Heather Lewandowski, a physicist at the University of Colorado (CU), Boulder. At her university this fall, lab exercises as diverse as building an electrical circuit or analyzing solar flare data will most likely be completely remote. Luckily, physics already had a foot in the virtual lab world—especially at CU. There, back in 2002, Nobel laureate Carl Wieman developed the Physics Education Technology (PhET) Interactive Simulations project to provide “games” that teach students basic physics concepts. The PhET web portal now has 106 physics-based simulations and another 50 or so for other disciplines. It became a go-to place this spring for faculty shifting to online teaching; traffic increased fivefold, says Director Katherine Perkins. In addition, several universities have adopted a handheld device called the iOLab that rents for $50 a semester. With it, students can measure magnetism, light intensity, acceleration, temperature, gravity, and atmospheric pressure, and do basic physics experiments at home. “They like that we trust them and are not just giving them instructions,” says iOLab inventor and physicist Mats Selen at the University of Illinois, Urbana-Champaign. Lewandowski and her colleagues surveyed physics instructors and students about their experiences and posted their findings on arXiv, the physics preprint server, on 2 July. Respondents said online labs work best when projects are open-ended, and online class meetings are kept small. They complained about technical difficulties, students having unequal access to the internet and materials, and longer prep times for both students and instructors. But they reported they could meet most key learning objectives, Lewandowski says, even though “there are lots of things we can't replicate in remote experiments,” such as such as building vacuum chambers or troubleshooting equipment. Some institutions decided this spring that virtual just wouldn't do. The Marine Biological Laboratory (MBL) in Woods Hole, Massachusetts, simply canceled its summer courses. “MBL courses are world-renowned for the intensity of the hands-on nature of the lab work,” says Director Nipam Patel. Students spend long hours with famous faculty and do their own projects using organisms collected locally. “We felt that it would be exceedingly difficult to replicate these experiences as a virtual lab course.” Other institutions will try for a mix of in-person and virtual labs. Suely Black, chemistry chair at Norfolk State University, expects only half of his students will be in lab each week this fall, while the other half will be in online classes analyzing data and writing reports. “The crisis has caused us to more critically evaluate what activities students must experience in the lab setting,” he says. Similarly, this fall, organic chemistry students at the University of Michigan (UM), Ann Arbor, will rotate into the lab in small groups, giving each a taste of the hands-on experience. Personal protection equipment is standard for this course and all the work is done in hoods with excellent air exchange, so “they are really fully protected,” says UM biochemist Kathleen Nolta. Storksdieck, an advocate of online learning, questions the value of smelling fumes or using a pipette. “We have to ask whether all the hands-on taught so far was all that great,” he says. Dominique Durand, a biomedical engineer at Case Western Reserve University, says after he put a master's program in biomedical engineering completely online 5 years ago, he concluded that solving problems was more important than hands-on experience. And University of California, Santa Cruz, ecologist Erika Zavaleta thinks virtual courses will open fieldwork to far more students. “There are things you can do online that you can't do in person,” she adds, such as visiting more places than possible by driving. Even so, Handschy laments that his geology students will not have the 12-hour-a-day immersive interactions with each other and faculty that past classes have had. Natalie White, a rising junior at UNH who took Moore's course on Appledore last year, agrees: “You don't have all the time in between when you walk around the island and can ask impromptu questions.” Appledore Island is the source of some her fondest memories. “I think they are missing out on the community.”
One of the major hurdles preventing the full exploitation of information from online communities is the widespread concern regarding the quality and credibility of user-contributed content. Prior works in this domain operate on a static snapshot of the community, making strong assumptions about the structure of the data (e.g., relational tables), or consider only shallow features for text classification. To address the above limitations, we propose probabilistic graphical models that can leverage the joint interplay between multiple factors in online communities --- like user interactions, community dynamics, and textual content --- to automatically assess the credibility of user-contributed online content, and the expertise of users and their evolution with user-interpretable explanation. To this end, we devise new models based on Conditional Random Fields for different settings like incorporating partial expert knowledge for semi-supervised learning, and handling discrete labels as well as numeric ratings for fine-grained analysis. This enables applications such as extracting reliable side-effects of drugs from user-contributed posts in healthforums, and identifying credible content in news communities. Online communities are dynamic, as users join and leave, adapt to evolving trends, and mature over time. To capture this dynamics, we propose generative models based on Hidden Markov Model, Latent Dirichlet Allocation, and Brownian Motion to trace the continuous evolution of user expertise and their language model over time. This allows us to identify expert users and credible content jointly over time, improving state-of-the-art recommender systems by explicitly considering the maturity of users. This also enables applications such as identifying helpful product reviews, and detecting fake and anomalous reviews with limited information.