Information retrieval

Information retrieval is defined as "a branch of computer or library science relating to the storage, locating, searching, and selecting, upon demand, relevant data on a given subject." As noted by Carl Sagan, "human beings have, in the most recent few tenths of a percent of our existence, invented not only extra-genetic but also extrasomatic knowledge: information stored outside our bodies, of which writing is the most notable example." The benefits of enhancing personal knowledge with retrieval of extrasomatic knowledge or transactive memory have been shown in comparisons with rote memory.

Although information retrieval is usually thought of being done by computer, retrieval can also be done by humans for other humans. In addition, some Internet search engines such as mahalo.com and http://www.chacha.com/ may have human supervision or editors.

Some Internet search engines such http://www.deeppeep.org and http://www.deepdyve.com/ as attempt to index the Deep Web which is web pages that are not normally public.

The usefulness of a search engine has been proposed to be:

${\mbox{Usefulness}}={\frac {{\mbox{Relevance}}*{\mbox{Validity}}}{\mbox{Work}}}$

Classification by user purpose

Information retrieval can be divided into information discovery, information recovery, and information awareness.

Information discovery

Information discovery is searching for information that the searcher has not seen before and the searcher does not know for sure that the information exists. Information discovery includes searching in order to answer a question at hand, or searching for a topic without a specific question in order to improve knowledge of a topic.

Information recovery

Information recovery is searching for information that the searcher has seen before and knows to exist.

Information awareness

Information awareness has also been described as "'systematic serendipity' - an organized process of information discovery of that which he [the searcher] did not know existed". Information awareness can be further divided into:

• Information familiarity
• Knowledge acquisition (or called recollection) is the ability to apply the new knowledge.

Examples of information awareness prior to the Internet include reading print and online periodicals. With the Internet, new methods include email newsletters, email alerts, and RSS feeds.

These methods may increase information familiarity.

Classification by indexing methods used

Document retrieval

Models for information retrieval of documents are based on either the text of the document or links to and from the document and other documents.

Models based on analysis of the text are the boolean, vector, and probabilistic.

Models based on analysis of the links include PageRank, HITS, and impact factor.

Boolean (set theoretic, exact matching)

Variants of the boolean model include:

• Fuzzy logic (used with thesauri)
• Extended boolean

Vector space model (relevancy, algebraic, partial match, ranking)

Relevancy is determined by weighting concept i in a document j by (tf-idf weighting):
${\text{Weight}}_{ij}={\text{Term frequency}}_{ij}\times {\text{Inverse document frequency}}_{ij}$

Where

${\text{Inverse document frequency}}_{i}={\text{log}}\left({\frac {\text{number of documents}}{{\text{number of documents with term }}i}}\right)+1$

and

${\text{Term frequency}}_{ij}={\text{log}}\left({\text{frequency of term }}i{\text{ in document }}j\right)$ Variants of the vector space model include:

• Generalized vector space model (allows for correlated search terms)
• Latent semantic indexing model (allows for search for synonymous concepts rather than literal search terms)
• Neural network model

Probabilistic (Bayes)

Variants of the probabilistic model include:

• Inference network
• Belief network

Factors associated with unsuccessful retrieval

The field of medicine provides much research on the difficulties of information retrieval. Barriers to successful retrieval include:

• Lack of prior experience with the information retrieval system being used
• Low visual spatial ability
• Poor formulation of the question to be searched
• Difficulty designing a search strategy when multiple resources are available
• "Uncertainty about how to know when all the relevant evidence has been found so that the search can stop"
• Difficulty synthesizing an answer across multiple documents

Factors associated with successful retrieval

Characteristics of how the information is stored

For storage of text content, the quality of the index to the content is important. For example, the use of stemming, or truncating, words by removing suffixes may help.

Display of information

Information that is structured may be more effective according to controlled studies. In addition, the structure should be layered with a summary of the content being the first layer that the readers sees. This allows the reader to take only an overview, or choose more detail. Some Internet search engines such as http://www.kosmix.com/ try to organize search results beyond a one dimensional list of results.

Regarding display of results from search engines, an interface designed to reduce anchoring and order bias may improve decision making.

Characteristics of the search engine

John Battelle has described features of the perfect search engine of the future. For example, the use of Boolean searching may not be as efficient. Meta-searching and task based searching may improve decision velocity.

Meta-search

Meta-search engines search multiple resources and integrate the results for the user. Examples in health care include Trip Database, MacPLUS, and QuickClinical.

Characteristics of the searcher

In healthcare, searchers are more likely to be successful if their answer is answer before searching, they have experience with the system they are searching, and they have a high spatial visualization score. Also in healthcare, physicians with less experience are more likely to want more information. Physicians who report stress when uncertain are more likely to search textbooks than source evidence.

In healthcare, using expert searchers on behalf of physicians led to increased satisfaction by the physicians with the search results.

Use of term overlap is associated with success.

Impact of information retrieval

The benefits of enhancing personal knowledge with retrieval of extrasomatic knowledge has been shown in a controlled comparison with rote memory. Various before and after comparisons are summarized in the tables.

Impact of medical searching by physicians and medical students

Search engine	Users	Questions	Portion of answers correct		Portion of answers that moved from correct to incorrect
			Before searching	After searching
Quick Clinical (federated search)	73 practicing doctors and clinical nurse consultants	Eight clinical questions 600 total responses	37%	50%	7%
User's own choice	23 primary care physicians	2 questions from a pool of 23 clinical questions from Hersh 46 total responses	39%	42%	11%
OVID	45 senior medical students (data available for nursing students)	5 questions from a pool of 23 clinical questions from Hersh 324 total responses	32%	52%	13%

Frequency that searching changed medical care.

	Searches	Frequency useful information found	Frequency changed care
Izcovich 2011	RCT of 407 inpatients compared to 402 control inpatients Searcher sought answers to questions that arose during "morning report". Search resources did not include UpToDate. Results emailed to teams.		No difference between study groups
Lucas 2011	Before after study of 146 inpatients Searcher sought answers to corroborate principle treated decisions for all patients. Search resources included UpToDate. Search results given to attendings. Blinded outcome assessment		• Treatments changed in 18% • Treatments improved in 14%
Crowley 2003	625 self-initiated searches, uncontrolled study	83%	39%
Rochester study 1992	uncontrolled study		80%
Chicago study 1987	questions searched by librarians in response to physician queries; uncontrolled study		74%

Critical incident studies can also document impact of information retrieval.

Evaluation of the quality of information retrieval

Survival curve modeling amount of time taken to answer questions. The units for time are arbitrary and meaningless in this example.

Logistic curve modeling rate of correct answers over time. The units for time are arbitrary and meaningless in this example.

Various methods exist to evaluate the quality of information retrieval. Hersh noted the classification of evaluation developed by Wancaster and Warner in which the first level of evaluation is:

• Costs/resources consumed in learning and using a system
• Time needed to use the system
• Quality of the results.
  • Coverage. An estimated of coverage can be crudely automated. However, more accurate judgment of relevance requires a human judge which introduces subjectivity.
  • Precision and recall
  • Novelty. This has been judged by independent reviewers.
  • Completeness and accuracy of results. An easy method of assessing this is to let the searcher make a subjective assessment. Other methods may be to use a bank of questions with known target documents or known answers.

• Usage
  • Self-reported
  • Measured

Precision and recall

Recall is the fraction of relevant documents that are successfully retrieved. This is the same as sensitivity. The recall has also been called the "yield" and comprehensiveness.

${\mbox{recall}}={\frac {|\{{\mbox{relevant documents}}\}\cap \{{\mbox{retrieved documents}}\}|}{|\{{\mbox{relevant documents}}\}|}}$

Precision is the fraction of retrieved documents that are relevant to the search. Precision has also been called efficiency. This is the same as positive predictive value.

${\mbox{precision}}={\frac {|\{{\mbox{relevant documents}}\}\cap \{{\mbox{retrieved documents}}\}|}{|\{{\mbox{retrieved documents}}\}|}}$

F₁ is the unweighted harmonic mean of the recall and precision.

Number needed to read

The number Needed to Read (NNR) is "how many papers in a journal have to be read to find one of adequate clinical quality and relevance." Of note, the NNR has been proposed as a metric to help libraries to decide which journals to subscribe to. The NNR has also been called the "burden."

Number needed to search

The humber needed to search (NNS) is the number of questions that would have to be searched for one question to be well answered.

Hit curve

A hit curve is the number of relevant documents retrieved among the first n results.

Decision velocity

Time need to answer a question can be compared between two systems with a Kaplan-Meir survival analysis method.

In health care, difficult questions make take hours to answer.

If the correct answer to the search question is known, a logistic function can model rate of correct answers over time. The result is an S-curve (also called sigmoid curve or logistic growth curve) in which most questions are answered after an initial delay; however, a minority of questions take a much longer time.

Critical incidents

Analysis of critical incidents may help.

Footnotes

In addition:

• Berthier Ribeiro-Neto; Ricardo Baeza-Yates; Ribeiro, Berthier de Araújo Neto (2009). Modern information retrieval. Boston: Addison-Wesley. ISBN 0-321-41691-0. 
• Shortliffe, Edward Hance; Cimino, James D. (2006). Biomedical informatics: computer applications in health care and biomedicine. Berlin: Springer. ISBN 0-387-28986-0.