Help for package LitFetchR

Type:

Package

Title:

Automatically Fetching References Metadata from Literature Databases

Version:

1.1.0

Maintainer:

Thomas Dumond <thomas.dumond@adelaide.edu.au>

Description:

Provides functions to automatically retrieve and deduplicate reference metadata based on saved search strings. Access to Web of Science and Scopus requires personal API keys, while PubMed can be queried without one. The optional deduplication functionality requires the package 'ASySD' available from https://github.com/camaradesuk/ASySD.

License:

MIT + file LICENSE

Encoding:

UTF-8

Imports:

dplyr, httr, jsonlite, openxlsx, purrr, xml2

Suggests:

ASySD, cronR, knitr, rmarkdown, testthat (≥ 3.0.0)

RoxygenNote:

7.3.3

Config/testthat/edition:

URL:

https://github.com/thomasdumond/LitFetchR, https://thomasdumond.github.io/LitFetchR/

BugReports:

https://github.com/thomasdumond/LitFetchR/issues

VignetteBuilder:

knitr

NeedsCompilation:

Packaged:

2026-11-12 04:18:53 UTC; a1224158

Author:

Thomas Dumond

[cre, aut, cph], Charles Caraguel

[ctb], Torben Nielsen

[ctb]

Repository:

CRAN

Date/Publication:

2026-11-12 16:00:09 UTC

Automating the retrieval of references based on a saved search string(s).

Description

Creates a read-only Rscript and a task to run the code automatically at a specified frequency and time, to retrieve references corresponding to the saved search string(s) on up to three platforms (e.g. Web of Science, Scopus and PubMed).

Usage

auto_LitFetchR_setup(
  task_id = "task_id",
  when = "DAILY",
  time = "08:00",
  wos = FALSE,
  scp = FALSE,
  pmd = FALSE,
  directory,
  dedup = FALSE,
  open_file = FALSE,
  dry_run = FALSE,
  day = NULL,
  output_format = c("csv", "ris")
)

Arguments

task_id

Name of the automated reference retrieval task (e.g. one keyword describing your review). Spaces are replaced by "-". A task with the same name must not exist already: use remove_scheduled_task() first to replace it.

when

Frequency of the automated reference retrieval task (DAILY, WEEKLY or MONTHLY).

time

Time of the automated reference retrieval task (must be HH:MM 24-hour clock format).

wos

Runs the search on Web of Science (TRUE or FALSE).

scp

Runs the search on Scopus (TRUE or FALSE).

pmd

Runs the search on PubMed (TRUE or FALSE).

directory

Choose the directory in which the search string is saved (Project's directory). That is also where the references metadata will be saved.

dedup

Deduplicates the retrieved references (TRUE or FALSE).

open_file

Automatically opens the output file after reference retrieval.

dry_run

Simulation run option.

day

Day on which a WEEKLY or MONTHLY task runs. For WEEKLY, one of MON, TUE, WED, THU, FRI, SAT or SUN. For MONTHLY, a day of the month between 1 and 28. If NULL (default), the weekday (WEEKLY) or day of the month (MONTHLY) of today's date is used. Ignored for DAILY.

output_format

Format of the file of references written at each run: "csv" (default) or "ris" (see manual_fetch()). The choice is fixed in the script of the task. To change it, remove the task with remove_scheduled_task() and create it again.

Details

The function reports "Task scheduled!" only after checking that the scheduler accepted the task and that the task exists. On Windows the registered task is read back from Task Scheduler and compared with the requested schedule. A short summary (recurrence, next run time and status) is then printed. If the scheduler returns an error, the function stops and shows the scheduler's own message. If the task cannot be confirmed, a warning is given instead of the success message. Check the summary: it is the only evidence that automatic retrieval will run.

The scheduled task runs only while the computer is on and you are logged in. Each run is recorded in run_log.csv (see manual_fetch()) with the name of the task in the column run_mode. When a run fails, or finishes with problems on one platform, it ends with a non-zero exit status, which the Windows Task Scheduler shows as the Last Run Result of the task.

Value

NULL (invisibly). Called for its side effects: writes an R script and schedules a task (Windows Task Scheduler or cron) to run the script automatically. The function stops with the scheduler's error message if the task cannot be created, and gives a warning if the new task cannot be confirmed.

Examples

# This is a "dry run" example.
# No task will actually be scheduled,
# it only shows how the function should react.
auto_LitFetchR_setup(task_id = "fish_vibrio",
                       when = "WEEKLY",
                       time = "14:00",
                       day = "MON",
                       wos = TRUE,
                       scp = TRUE,
                       pmd = TRUE,
                       directory,
                       dedup = FALSE,
                       open_file = FALSE,
                       dry_run = TRUE
                       )

Builds the query submitted to a literature platform.

Description

Takes the search string entered by the user and returns the exact query text sent to the API of the selected platform, before URL encoding. The field tag of the platform is added (PubMed: none, Scopus: TITLE-ABS-KEY, Web of Science: TS) and the exclusion operator is written the way each platform needs it, outside quoted phrases: ⁠AND NOT⁠ for Scopus and NOT for PubMed and Web of Science (PubMed ignores the NOT of an ⁠AND NOT⁠ and returns the records to be excluded, and Web of Science rejects it). This is the only place where the search string is modified, so that the counts shown by create_save_search() and the records retrieved by manual_fetch() always use the same query.

Usage

build_platform_query(search, platform = c("wos", "scp", "pmd"))

Arguments

search

Search string entered by the user (single character string).

platform

One of "wos", "scp" or "pmd".

Value

A single character string, not URL encoded.

Creates a unique name for any document.

Description

Creates a unique name for any document.

Usage

build_sheet_name(time = Sys.time())

Arguments

time

System time at the time the function is run.

Value

Character scalar. A unique name based on the system time with the format YYYY-MM-DD-HHMMSS.

Creates an excel file to store the deduplication history.

Description

Creates an excel file to store the deduplication history.

Usage

create_dedup_history(directory)

Arguments

directory

Choose the directory in which the references deduplication history will be saved.

Value

A list with elements:

history_dedup: A Workbook object (from openxlsx).
hist_dedup_path: Character. Path to the created .xlsx file.

Creates an excel file to store the references identification retrieved at each search.

Description

Creates an excel file to store the references identification retrieved at each search.

Usage

create_id_history(directory)

Arguments

directory

Choose the directory in which the references identification history will be saved.

Value

A list with element:

history_id: A Workbook object (from openxlsx).

Creates and saves search string(s).

Description

An interactive function that ask the user to enter a search string and provide the number of results from 3 platforms: Web of Science, Scopus and PubMed. You can then save one or more search strings to retrieve the references later.

Usage

create_save_search(
  wos = FALSE,
  scp = FALSE,
  pmd = FALSE,
  directory,
  dry_run = FALSE
)

Arguments

wos

Runs the search on Web of Science (TRUE or FALSE).

scp

Runs the search on Scopus (TRUE or FALSE).

pmd

Runs the search on PubMed (TRUE or FALSE).

directory

Choose the directory in which the search string and the search history will be saved.

dry_run

Simulation run option.

Details

Type summary to display the searches run so far and save one of them (the saved search strings are stored in "search_list.txt", which is required by the retrieval functions). Type exit to leave the tool. If searches were run but none was saved during the session, exit first offers to save one.

Value

NULL (invisibly). Called for its side effects: interactive querying and writing search history files.

Examples

# This is a "dry run" example.
# No search will be created and no database will be accessed.
# It only shows how the function should react.
create_save_search(wos = TRUE,
                   scp = TRUE,
                   pmd = TRUE,
                   directory,
                   dry_run = TRUE)

Creates an excel file to store the history of searches made using `create_save_search()`.

Description

Creates an excel file to store the history of searches made using create_save_search().

Usage

create_search_history(directory)

Arguments

directory

Choose the directory in which the search history will be saved.

Value

A list with elements:

history_search: Workbook object.
sheet_name: Character scalar.

Deduplicates the references from up to three dataframes.

Description

Deduplicates the references from up to three dataframes.

Usage

dedup_refs(
  df1 = NULL,
  df2 = NULL,
  df3 = NULL,
  directory,
  open_file = FALSE,
  dry_run = FALSE,
  output_format = c("csv", "ris")
)

Arguments

df1

Dataframe 1 (can be NULL)

df2

Dataframe 2 (can be NULL)

df3

Dataframe 3 (can be NULL)

directory

Choose the directory in which the references deduplication history will be saved.

open_file

Automatically opens the output file after reference retrieval.

dry_run

Simulation run option.

output_format

Format of the file of deduplicated references: "csv" (default) or "ris" (see manual_fetch()).

Value

NULL (invisibly). Called for its side effects: writes a CSV or RIS file of deduplicated citations and an Excel workbook recording the deduplication history.

Examples

# This is a "dry run" example.
# No deduplication will happen.
# It only shows how the function should react.
dedup_refs(df1 = df_vibrio_wos,
           df2 = df_vibrio_scp,
           df3 = df_vibrio_pmd,
           directory = tempdir(),
           open_file = FALSE,
           dry_run = TRUE
           )

Extracts the metadata from the new references found on PubMed based on the search string(s) saved in "search_list.txt".

Description

Extracts the metadata from the new references found on PubMed based on the search string(s) saved in "search_list.txt".

Usage

extract_pmd_list(
  search_list_path,
  directory,
  run_id = NULL,
  commit_history = TRUE
)

Arguments

search_list_path

Path to "search_list.txt".

directory

Choose the directory in which the references identification history will be saved.

run_id

Identifier shared by all the rows of one retrieval in "run_log.csv" (created if NULL).

commit_history

If TRUE (default), the references identification history and the run log are written before returning. If FALSE, nothing is written: the pending history update and the run log rows are attached to the returned data frame (attributes pending_history and run_log) so that the caller can write them once the output file exists (used by manual_fetch()).

Value

A data.frame with one row per retrieved PubMed record and columns:

author: Character. Publication authors.
year: Character. Publication year.
title: Character. Publication title.
journal: Character. Publication journal name.
volume: Character. Publication journal volume.
number: Character. Publication journal issue number.
abstract: Character. Publication abstract.
doi: Character. Publication Digital Object Identifier (DOI).
pages: Character. Publication page range (e.g. "179-192").
isbn: Character. ISBN for book chapters (NA for journal articles).
source: Character. Data source.
platform_id: Character. Publication unique identifier in data source.

If search_list_path does not exist, returns NULL. If any search fails, an error of class litfetchr_platform_error is raised and nothing is added to the history.

Extracts the metadata from the new references found on Scopus based on the search string(s) saved in "search_list.txt".

Description

Extracts the metadata from the new references found on Scopus based on the search string(s) saved in "search_list.txt".

Usage

extract_scp_list(
  search_list_path,
  directory,
  run_id = NULL,
  commit_history = TRUE
)

Arguments

search_list_path

Path to "search_list.txt".

directory

Choose the directory in which the references identification history will be saved.

run_id

Identifier shared by all the rows of one retrieval in "run_log.csv" (created if NULL).

commit_history

Value

A data.frame with one row per retrieved Scopus record and columns:

author: Character. Publication authors.
year: Character. Publication year.
title: Character. Publication title.
journal: Character. Publication journal name.
volume: Character. Publication journal volume.
number: Character. Publication journal issue number.
abstract: Character. Publication abstract.
doi: Character. Publication Digital Object Identifier (DOI).
pages: Character. Publication page range (e.g. "179-192").
isbn: Character. ISBN for book chapters (NA for journal articles).
source: Character. Data source.
platform_id: Character. Publication unique identifier in data source.

If search_list_path does not exist, returns NULL. If any search fails (or the API key is missing), an error of class litfetchr_platform_error is raised and nothing is added to the history.

extract the metadata from the new references from Web of Science based on the search strings found in search_list.txt

Description

extract the metadata from the new references from Web of Science based on the search strings found in search_list.txt

Usage

extract_wos_list(
  search_list_path,
  directory,
  run_id = NULL,
  commit_history = TRUE
)

Arguments

search_list_path

path to search_list

directory

Choose the directory in which the references identification history will be saved.

run_id

Identifier shared by all the rows of one retrieval in "run_log.csv" (created if NULL).

commit_history

Value

A data.frame with one row per retrieved Web of Science record and columns:

author: Character. Publication authors.
year: Character. Publication year.
title: Character. Publication title.
journal: Character. Publication journal name.
volume: Character. Publication journal volume.
number: Character. Publication journal issue number.
abstract: Character. Publication abstract.
doi: Character. Publication DOI, or article URL when DOI is unavailable.
pages: Character. Publication page range (e.g. "179-192").
isbn: Character. ISBN for book chapters (NA for journal articles).
source: Character. Data source.
platform_id: Character. Publication unique identifier in data source.

If search_list_path does not exist, returns NULL. If any search fails (or the API key is missing), an error of class litfetchr_platform_error is raised and nothing is added to the history.

Transforms a long computer path into a shorter.

Description

Transforms a long computer path into a shorter.

Usage

get_short_path(path)

Arguments

path

Path to the document.

Value

Character scalar, a shorter path to use in Windows OS.

Manual literature retrieval.

Description

Retrieves references corresponding to the saved search string(s) on up to three platforms (e.g. Web of Science, Scopus and PubMed).

Usage

manual_fetch(
  wos = FALSE,
  scp = FALSE,
  pmd = FALSE,
  directory,
  dedup = FALSE,
  open_file = FALSE,
  dry_run = FALSE,
  output_format = c("csv", "ris")
)

Arguments

wos

Runs the search on Web of Science (TRUE or FALSE).

scp

Runs the search on Scopus (TRUE or FALSE).

pmd

Runs the search on PubMed (TRUE or FALSE).

directory

Choose the directory in which the search string is saved (Project's directory). That is also where the references metadata will be saved.

dedup

Deduplicates the retrieved references (TRUE or FALSE).

open_file

Automatically opens the output file after reference retrieval.

dry_run

Simulation run option.

output_format

Format of the file of references: "csv" (default) or "ris". RIS is read by most reference managers and screening tools, for example Covidence, which does not accept CSV files. A RIS file has one AU line per author and always describes journal articles. With dedup = TRUE the deduplicated references are written in the chosen format.

Details

Every retrieval is recorded in the file run_log.csv of directory (one row per saved search and platform, or one row with platform "all" if the run could not start, for example because search_list.txt is invalid). It records whether the run was started by hand or by a scheduled task (run_mode). It stores the exact query submitted to each platform, the date, the package and R versions, the number of results and new records, the status of the request and any error message. For PubMed, it also stores the query translation reported by PubMed. For Web of Science, it also stores the number of records retrieved from each source database.

The history of the references already retrieved (history_id.xlsx) is only updated once the output file has been written. If a platform fails (for example an expired API key), the other platforms are still saved, a warning names the failed platform(s), and the failed platform is not added to the history, so its references are retrieved again by the next run. If all the selected platforms fail, an error is raised. When there is no new record, no output file is written.

Value

NULL (invisibly). Called for its side effects: Create a CSV or RIS file with the references metadata, a history file of the references retrieved, a history file of the deduplication (if the option is selected) and a run log (run_log.csv).

Examples

# This is a "dry run" example.
# No references will actually be scheduled, it only shows how the function should react.
manual_fetch(wos = TRUE,
             scp = TRUE,
             pmd = TRUE,
             directory,
             dedup = TRUE,
             open_file = FALSE,
             dry_run = TRUE
             )

Reads and validates the saved search strings.

Description

Reads "search_list.txt" and returns the saved searches as a named character vector. The file is parsed in the same way as in the extraction functions: blank lines are dropped, the name is the text before the first = and the query is everything after the first = (so a query may itself contain =). The function stops with an informative message when the file is missing, has no search, or is malformed (a line without =, an empty name or query, or duplicated names).

Usage

read_search_list(path, allow_empty = FALSE)

Arguments

path

Path to "search_list.txt".

allow_empty

If TRUE, a file without any search returns an empty named character vector instead of an error. Defaults to FALSE.

Value

A named character vector. The names are the search names and the values are the query strings.

Removes a scheduled task.

Description

Removes a scheduled task using the "task_id" from Task Scheduler (Windows) or Cron (Mac/Linux).

Usage

remove_scheduled_task(task_id, dry_run = FALSE)

Arguments

task_id

Name/ID of the scheduled task (Windows Task Scheduler or Cron).

dry_run

Simulation run option.

Details

The function checks whether the task exists first, removes it, then checks that it is gone. It reports which of these happened. If the task does not exist, nothing is removed and a message says so. If the scheduler returns an error, or the task is still there afterwards, the function stops and shows the scheduler's own message.

Value

NULL (invisibly). Called for its side effects: removes a scheduled task saved using the function 'auto_LitFetchR_setup'.

Examples

# This is a "dry run" example.
# No task will actually be removed, it only shows how the function should react.
remove_scheduled_task("fish_vibrio",
                      dry_run = TRUE
                      )

Saves Web of Science, Scopus, and/or NCBI/PubMed API keys in .Renviron.

Description

You can set wos_api_key, scp_api_key, scp_insttoken, ncbi_api_key, or any combination. Remember to restart the R session after saving your API keys.

Usage

save_api_keys(
  wos_api_key = NULL,
  scp_api_key = NULL,
  scp_insttoken = NULL,
  ncbi_api_key = NULL,
  dry_run = FALSE
)

Arguments

wos_api_key

The API key value for Web of Science (use quotation marks).

scp_api_key

The API key value for Scopus (use quotation marks).

scp_insttoken

The institutional token for Scopus (use quotation marks). Required by some institutional Scopus subscriptions alongside the API key.

ncbi_api_key

The API key value for NCBI/PubMed (use quotation marks). Optional; when set, PubMed requests run at a higher rate limit.

dry_run

Simulation run option.

Value

Logical. TRUE if at least one value was written, FALSE if left unchanged.

Examples

save_api_keys(wos_api_key = "abcd01234",
               scp_api_key = "efgh5678",
               dry_run = TRUE
               )

Package {LitFetchR}

Automating the retrieval of references based on a saved search string(s).

Description

Usage

Arguments

Details

Value

Examples

Builds the query submitted to a literature platform.

Description

Usage

Arguments

Value

Creates a unique name for any document.

Description

Usage

Arguments

Value

Creates an excel file to store the deduplication history.

Description

Usage

Arguments

Value

Creates an excel file to store the references identification retrieved at each search.

Description

Usage

Arguments

Value

Creates and saves search string(s).

Description

Usage

Arguments

Details

Value

Examples

Creates an excel file to store the history of searches made using create_save_search().

Description

Usage

Arguments

Value

Deduplicates the references from up to three dataframes.

Description

Usage

Arguments

Value

Examples

Extracts the metadata from the new references found on PubMed based on the search string(s) saved in "search_list.txt".

Description

Usage

Arguments

Value

Extracts the metadata from the new references found on Scopus based on the search string(s) saved in "search_list.txt".

Description

Usage

Arguments

Value

extract the metadata from the new references from Web of Science based on the search strings found in search_list.txt

Description

Usage

Arguments

Value

Transforms a long computer path into a shorter.

Description

Usage

Arguments

Value

Manual literature retrieval.

Description

Usage

Arguments

Details

Value

Examples

Reads and validates the saved search strings.

Description

Usage

Arguments

Value

Removes a scheduled task.

Description

Creates an excel file to store the history of searches made using `create_save_search()`.