Title:	AI Screening Tools in R for Systematic Reviewing
Version:	0.3.1
Description:	Provides functions to conduct title and abstract screening in systematic reviews using large language models, such as the Generative Pre-trained Transformer (GPT) models from 'OpenAI' https://developers.openai.com/. These functions can enhance the quality of title and abstract screenings while reducing the total screening time significantly. In addition, the package includes tools for quality assessment of title and abstract screenings, as described in Vembye, Christensen, Mølgaard, and Schytt (2025) <doi:10.1037/met0000769>.
Depends:	R (≥ 4.1.0)
License:	GPL (≥ 3)
Encoding:	UTF-8
RoxygenNote:	7.3.3
Imports:	dplyr, tibble, httr2, stringr, furrr, tidyr, tictoc, askpass, curl, purrr, lifecycle, jsonlite, htmltools, tidyselect, rlang
Suggests:	future, knitr, rmarkdown, usethis, testthat (≥ 3.0.0), withr, readtext, quarto
Config/testthat/edition:	3
Config/testthat/parallel:	true
Config/testthat/start-first:	tabscreen_gpt, approximate_price_gpt, rate_limits, api_key_functions
URL:	https://mikkelvembye.github.io/AIscreenR/, https://github.com/MikkelVembye/AIscreenR
BugReports:	https://github.com/MikkelVembye/AIscreenR/issues
LazyData:	true
VignetteBuilder:	quarto
Config/Needs/website:	quarto
NeedsCompilation:	no
Packaged:	2026-04-13 16:21:21 UTC; B199526
Author:	Mikkel H. Vembye [aut, cre], Thomas Olsen [aut]
Maintainer:	Mikkel H. Vembye <mikkel.vembye@gmail.com>
Repository:	CRAN
Date/Publication:	2026-04-13 19:20:14 UTC

AIscreenR: AI Screening Tools in R for Systematic Reviewing

Description

Provides functions to conduct title and abstract screening in systematic reviews using large language models, such as the Generative Pre-trained Transformer (GPT) models from 'OpenAI' https://developers.openai.com/. These functions can enhance the quality of title and abstract screenings while reducing the total screening time significantly. In addition, the package includes tools for quality assessment of title and abstract screenings, as described in Vembye, Christensen, Mølgaard, and Schytt (2025) doi:10.1037/met0000769.

Author(s)

Maintainer: Mikkel H. Vembye mikkel.vembye@gmail.com (ORCID)

Authors:

• Thomas Olsen thomasolsen123@hotmail.com

See Also

Useful links:

• https://mikkelvembye.github.io/AIscreenR/

• https://github.com/MikkelVembye/AIscreenR

• Report bugs at https://github.com/MikkelVembye/AIscreenR/issues

Approximate price estimation for title and abstract screening using OpenAI's GPT API models

Description

This function supports the approximation of the price of title and abstract screenings when using OpenAI's GPT API models. The function only provide approximately accurate price estimates. When detailed descriptions are used, this will increase the completion tokens with an unknown amount.

Usage

approximate_price_gpt(
  data,
  prompt,
  studyid,
  title,
  abstract,
  model = "gpt-4o-mini",
  reps = 1,
  top_p = 1,
  token_word_ratio = 1.6,
  reasoning_effort = "medium",
  verbosity = "low"
)

Arguments

Value

An object of class "gpt_price". The object is a list containing the following components:

Examples

prompt <- "This is a prompt"

app_price <- approximate_price_gpt(
  data = filges2015_dat[1:2,],
  prompt = prompt,
  studyid = studyid,
  title = title,
  abstract = abstract,
  model = c("gpt-4o-mini", "gpt-4"),
  reps = c(10, 1)
)

app_price
app_price$price_dollar
app_price$price_data

Function to generate dataset to be used for fine-tuning models

Description

This function creates the initial data that can be used to fine tune models from OpenAI.

Usage

create_fine_tune_data(data, prompt, studyid, title, abstract)

Arguments

Value

A dataset of class 'fine_tune_data'.

Note

The dataset contains at least the following variables:

See Also

save_fine_tune_data()

Examples

# Extract 5 irrelevant and relevant records, respectively.
dat <- filges2015_dat[c(1:5, 261:265),]

prompt <- "Is this study about functional family therapy?"

dat <-
  create_fine_tune_data(
    data = dat,
    prompt = prompt,
    studyid = studyid,
    title = title,
    abstract = abstract
   )

dat

Disagreement sample data

Description

Example rows where human screening decisions differ from GPT decisions. Each row is a (study × prompt) screening outcome.

Usage

disagreements

Format

A tibble/data.frame with one row per screened (studyid, promptid) and 17 columns:

RIS file data from Functional Family Therapy (FFT) systematic review

Description

Bibliometric toy data from a systematic review regarding Functional Family Therapy (FFT) for Young People in Treatment for Non-opioid Drug Use (Filges et al., 2015). The data includes all 90 included and 180 excluded randomly sampled references from the literature search of the systematic review.

Usage

filges2015_dat

Format

A tibble with 270 rows/studies and 6 variables/columns

References

Filges, T., Andersen, D, & Jørgensen, A-M. K (2015). Functional Family Therapy (FFT) for Young People in Treatment for Non-opioid Drug Use: A Systematic Review Campbell Systematic Reviews, doi:10.4073/csr.2015.14

Get API key from R environment variable.

Description

Get API key from R environment variable.

Usage

get_api_key(env_var = "OPENAI_API_KEY")

Arguments

Details

get_api_key() can be used after executing set_api_key() or by adding the api key permanently to your R environment by using usethis::edit_r_environ(). Then write ⁠OPENAI_API_KEY=[insert your api key here]⁠ and close the .Renviron window and restart R. For backward compatibility, it will also check CHATGPT_KEY if OPENAI_API_KEY is not set.

Value

The specified API key (NOTE: Avoid exposing this in the console).

Note

Find your personal API key via the OpenAI quickstart guide at https://developers.openai.com/api/docs/quickstart#generate-an-api-key.

See Also

set_api_key.

Examples

## Not run: 
get_api_key()

## End(Not run)

Get GROQ API key from R environment variable.

Description

Get GROQ API key from R environment variable.

Usage

get_api_key_groq(env_var = "GROQ_API_KEY")

Arguments

Details

get_api_key_groq() can be used after executing set_api_key() or by adding the api key permanently to your R environment by using usethis::edit_r_environ(). Then write ⁠GROQ_API_KEY=[insert your api key here]⁠ and close the .Renviron window and restart R.

Value

The specified API key (NOTE: Avoid exposing this in the console).

Note

Find your personal API key at https://console.groq.com/keys.

See Also

set_api_key.

Examples

## Not run: 
get_api_key_groq()

## End(Not run)

Groq model prices (last updated March 18, 2026)

Description

Data set containing input and output prizes for all GROQ's API models.

Usage

groq_model_prizes

Format

A data.frame containing 4 rows/models and 3 variables/columns

References

GROQ. Pricing. https://groq.com/pricing

Test if the object is a 'chatgpt' object

Description

This function returns TRUE for chatgpt objects, and FALSE for all other objects.

Usage

is_chatgpt(x)

Arguments

Value

TRUE if the object inherits from the chatgpt class.

Test if the object is a 'chatgpt_tbl' object

Description

This function returns TRUE for chatgpt_tbl objects, and FALSE for all other objects.

Usage

is_chatgpt_tbl(x)

Arguments

Value

TRUE if the object inherits from the chatgpt_tbl class.

Test if the object is a 'gpt' object

Description

This function returns TRUE for gpt objects, and FALSE for all other objects.

Usage

is_gpt(x)

Arguments

Value

TRUE if the object inherits from the gpt class.

Test if the object is a 'gpt_agg_tbl' object

Description

This function returns TRUE for gpt_agg_tbl objects, and FALSE for all other objects.

Usage

is_gpt_agg_tbl(x)

Arguments

Value

TRUE if the object inherits from the gpt_agg_tbl class.

Test if the object is a 'gpt_tbl' object

Description

This function returns TRUE for gpt_tbl objects, and FALSE for all other objects.

Usage

is_gpt_tbl(x)

Arguments

Value

TRUE if the object inherits from the gpt_tbl class.

Model prize data (last updated March 18, 2026)

Description

Dataset mainly containing input and output prizes for all OpenAI's GPT API models.

Usage

model_prizes

Format

A data.frame containing 36 rows/models and 3 variables/columns

References

OpenAI. Pricing. https://developers.openai.com/api/docs/pricing

Print methods for 'chatgpt' objects

Description

Print methods for 'chatgpt' objects

Usage

## S3 method for class 'chatgpt'
print(x, ...)

Arguments

Value

Information about how to find answer data sets and pricing information.

Examples

## Not run: 
print(x)

## End(Not run)

Print methods for 'gpt' objects

Description

Print methods for 'gpt' objects

Usage

## S3 method for class 'gpt'
print(x, ...)

Arguments

Value

Information about how to find answer data sets and pricing information.

Examples

## Not run: 
print(x)

## End(Not run)

Print methods for 'gpt_price' objects

Description

Print methods for 'gpt_price' objects

Usage

## S3 method for class 'gpt_price'
print(x, ...)

Arguments

Value

The total price of the screening across all gpt-models expected to be used for the screening.

Examples

## Not run: 
print(x)

## End(Not run)

Print method for 'groq' objects

Description

Print method for 'groq' objects

Usage

## S3 method for class 'groq'
print(x, ...)

Arguments

Value

Information about how to find answer data sets and pricing information.

Examples

## Not run: 
print(x)

## End(Not run)

Find updated rate limits for API models

Description

rate_limits_per_minute reports the rate limits for a given API model. The function returns the available requests per minute (RPM) as well as tokens per minute (TPM). Find general information at https://developers.openai.com/api/docs/models/model-endpoint-compatibility.

Usage

rate_limits_per_minute(
  model = "gpt-4o-mini",
  AI_tool = "OpenAI",
  api_key = NULL
)

Arguments

Value

A tibble including variables with information about the model used, the number of requests and tokens per minute.

Examples

## Not run: 
set_api_key()

rate_limits_per_minute(
  model = "gpt-4o-mini",
  AI_tool = "OpenAI",
  api_key = get_api_key()
)

# Groq example
rate_limits_per_minute(
  model = "llama3-70b-8192",
  AI_tool = "Groq",
  api_key = get_api_key_groq()
)

## End(Not run)

Read an RIS file into a data frame

Description

Parses an RIS file into a data.frame, preserving the order of tags as they first appear in the file. Repeated tags within a record are collapsed into a single semicolon-separated string.

Usage

read_ris_to_dataframe(file_path)

Arguments

Value

A data.frame with one row per record and one column per encountered RIS tag, using descriptive column names. Columns are ordered by first appearance of the tag in the file. Repeated tag values are collapsed with "; ".

Examples

## Not run: 
df <- read_ris_to_dataframe("data-raw/raw data/apa_psycinfo_test_data.ris")

## End(Not run)

Generate a report for screening disagreements between human and AI decisions

Description

This function generates a report for screening disagreements between human and GPT decisions. It extracts information from the provided data. The function then compiles this information into a report using Quarto. The report can be saved in formats as HTML, PDF, Word, or more. The generated report includes sections for each study, displaying the study ID, title, abstract, and the decision generated by the GPT API model. The report also includes a section for a comment on the GPT decision, which the user will provide. The function also provides options to customize the document title, subtitle, and output directory.

Usage

report(
  data,
  studyid,
  title,
  abstract,
  gpt_answer,
  human_code,
  final_decision_gpt_num,
  file,
  format = "html",
  open = TRUE,
  document_title,
  document_subtitle = "",
  directory = getwd()
)

Arguments

Value

An object of class 'report'. The object is a list containing the following components:

Examples

## Not run: 
# Generate a report from the disagreements data
report(
  data = disagreements,
  studyid = studyid,
  title = title,
  abstract = abstract,
  gpt_answer = longest_answer,
  human_code = human_code,
  final_decision_gpt_num = final_decision_gpt_num,
  file = "Screening_Disagreements_Report",
  format = "html",
  document_title = "Study Report - Disagreement Explanations",
  open = TRUE
)

## End(Not run)

Random sample references

Description

sample_referencessamples n rows from the dataset with titles and abstracts either with or without replacement. This function is supposed to support the construct of a test dataset, as suggested by Vembye et al. (2025).

Usage

sample_references(
  data,
  n,
  with_replacement = FALSE,
  prob_vec = rep(1/n, nrow(data))
)

Arguments

Value

A dataset with n rows.

References

Vembye, M. H., Christensen, J., Mølgaard, A. B., & Schytt, F. L. W. (2025). Generative Pretrained Transformer Models Can Function as Highly Reliable Second Screeners of Titles and Abstracts in Systematic Reviews: A Proof of Concept and Common Guidelines. Psychological Methods. doi:10.1037/met0000769

Examples

excl_test_dat <- filges2015_dat[1:200,] |> sample_references(100)

Write a data frame to a RIS file

Description

Writes a data.frame to a RIS file, one record per row. If the data frame was created by read_ris_to_dataframe(), the original RIS tag order and tags are preserved where possible. Otherwise, a standard RIS format is used.

Usage

save_dataframe_to_ris(df, file_path)

Arguments

Details

If a field value contains semicolons, it is split and written as multiple tag lines. The TY (source type) field is written first for each record, followed by all other fields. Records are terminated with ER - .

Value

A character string indicating the file path where the RIS file was saved.

Examples

## Not run: 
df <- read_ris_to_dataframe("data-raw/raw data/apa_psycinfo_test_data.ris")
save_dataframe_to_ris(df, "path/to/output.ris")

## End(Not run)

Function to write/save fine tune dataset in required jsonl format

Description

This function creates jsonl training data that can be used to fine tune models from OpenAI. To generate a fine tuned model, this written data can be uploaded via https://developers.openai.com/api/docs/guides/supervised-fine-tuning.

Usage

save_fine_tune_data(
  data,
  role_and_subject,
  file,
  true_answer,
  roles = c("system", "user", "assistant")
)

Arguments

Value

A jsonl dataset to the set working directory.

See Also

create_fine_tune_data()

Examples

# Extract 5 irrelevant and relevant records, respectively.
library(dplyr)

dat <- filges2015_dat[c(1:5, 261:265),]

prompt <- "Is this study about functional family therapy?"

ft_dat <-
  create_fine_tune_data(
    data = dat,
    prompt = prompt,
    studyid = studyid,
    title = title,
    abstract = abstract
    ) |>
    mutate(true_answer = if_else(human_code == 1, "Include", "Exclude"))

role_subject <- paste0(
  "Act as a systematic reviewer that is screening study titles and ",
  "abstracts for your systematic reviews regarding the the effects ",
  "of family-based interventions on drug abuse reduction for young ",
  "people in treatment for non-opioid drug use."
)

# Saving data in jsonl format (required format by OpenAI)
fil <- tempfile("fine_tune_data", fileext = ".jsonl")

save_fine_tune_data(
  data = ft_dat,
  role_and_subject = role_subject,
  file = fil
)

Analyze performance between the human and AI screening.

Description

When both the human and AI title and abstract screening has been done, this function allows you to calculate performance measures of the screening, including the overall accuracy, specificity/recall, and sensitivity of the screening, as well as inter-rater reliability kappa statistics (Gartlehner et al., 2019; McHugh, 2012; Syriani et al., 2024).

Usage

screen_analyzer(x, human_decision = human_code, key_result = TRUE)

Arguments

Value

A tibble with screening performance measures. The tibble includes the following variables:

References

Gartlehner, G., Wagner, G., Lux, L., Affengruber, L., Dobrescu, A., Kaminski-Hartenthaler, A., & Viswanathan, M. (2019). Assessing the accuracy of machine-assisted abstract screening with DistillerAI: a user study. Systematic Reviews, 8:277, 1-10. doi:10.1186/s13643-019-1221-3

McHugh, M. L. (2012). Interrater reliability: The kappa statistic. Biochemia Medica, 22(3), 276-282. https://pubmed.ncbi.nlm.nih.gov/23092060/

Syriani, E., David, I., & Kumar, G. (2023). Assessing the Ability of ChatGPT to Screen Articles for Systematic Reviews. ArXiv Preprint ArXiv:2307.06464.

Examples

## Not run: 

library(future)

set_api_key()

prompt <- "Is this study about a Functional Family Therapy (FFT) intervention?"

plan(multisession)

res <- tabscreen_gpt(
  data = filges2015_dat[1:2,],
  prompt = prompt,
  studyid = studyid,
  title = title,
  abstract = abstract
  )

plan(sequential)

res |> screen_analyzer()


## End(Not run)

Generic function to re-screen failed title and abstract requests.

Description

This is a generic function to re-screen failed title and abstract requests. It reuses the arguments captured during the original screening and only re-submits the rows stored in object$error_data to the appropriate backend.

Usage

screen_errors(
  object,
  api_key = NULL,
  max_tries = NULL,
  max_seconds = NULL,
  is_transient = NULL,
  backoff = NULL,
  after = NULL,
  studyid = NULL,
  title = NULL,
  abstract = NULL,
  ...
)

Arguments

Details

The backend is derived from class(object) and mapped to either tabscreen_gpt() or tabscreen_groq(). Only rows in object$error_data are re-submitted. To avoid name collisions during unnesting in the backend, columns that will be regenerated (currently decision_binary, decision_description, error_message, res) are dropped from error_data before the call. The original arguments from the first screening are taken from attr(object, "arg_list") and are combined with any non-NULL overrides provided here.

Value

An object of class 'gpt' or 'groq' similar to the object returned by the original screening function, with:

• answer_data updated to include newly successful rows,

• error_data updated to include only remaining failures. Other fields (e.g., price_data, price_dollar, and arg_list) are preserved or updated by the backend.

See Also

tabscreen_gpt(), tabscreen_groq()

Examples

## Not run: 

# Example with openai
set_api_key()
prompt <- "Is this study about a Functional Family Therapy (FFT) intervention?"

obj_with_error <-
  tabscreen_gpt(
    data = filges2015_dat[1:2,],
    prompt = prompt,
    studyid = studyid,
    title = title,
    abstract = abstract,
    model = "gpt-4o-mini"
    )

obj_rescreened <-
  obj_with_error |>
  screen_errors()


# Example with groq
prompt <- "Is this study about a Functional Family Therapy (FFT) intervention?"

obj_with_error <-
  tabscreen_groq(
    data = filges2015_dat[1:2,],
    prompt = prompt,
    studyid = studyid,
    title = title,
    abstract = abstract,
    model = "llama-3.3-70b-versatile"
  )

obj_rescreened <-
  obj_with_error |>
  screen_errors()

## End(Not run)

Re-screen failed requests.

Description

This function supports re-screening of all failed title and abstract requests screened with tabscreen_gpt.original(). This function has been deprecated because OpenAI has deprecated the function_call and functions argument that was used in tabscreen_gpt.original().

Usage

screen_errors.chatgpt(
  object,
  ...,
  api_key = get_api_key(),
  max_tries = 4,
  max_seconds,
  is_transient,
  backoff,
  after
)

Arguments

Value

Object of class 'chatgpt' similar to the object returned by tabscreen_gpt.original(). See documentation value for tabscreen_gpt.original().

References

Wickham H (2023). httr2: Perform HTTP Requests and Process the Responses. https://httr2.r-lib.org, https://github.com/r-lib/httr2.

See Also

tabscreen_gpt.original()

Examples

## Not run: 

set_api_key()

prompt <- "Is this study about a Functional Family Therapy (FFT) intervention?"

obj_with_error <-
  tabscreen_gpt(
    data = filges2015_dat[1:2,],
    prompt = prompt,
    studyid = studyid,
    title = title,
    abstract = abstract,
    model = c("gpt-3.5-turbo-0613", "gpt-3.5-turbo-16k-0613"),
    max_tries = 1,
    reps = 10
    )

obj_rescreened <-
  obj_with_error |>
  screen_error()

# Alternatively re-set max_tries if errors still appear
obj_rescreened <-
  obj_with_error |>
  screen_error(max_tries = 16)

## End(Not run)

Re-screen failed requests.

Description

This function supports re-screening of all failed title and abstract requests screened with tabscreen_gpt()/tabscreen_gpt.tools().

Usage

screen_errors.gpt(
  object,
  api_key = get_api_key(),
  max_tries = 16,
  max_seconds,
  is_transient,
  backoff,
  after,
  ...
)

Arguments

Value

An object of class 'gpt' similar to the object returned by tabscreen_gpt(). See documentation for tabscreen_gpt().

References

Wickham H (2023). httr2: Perform HTTP Requests and Process the Responses. https://httr2.r-lib.org, https://github.com/r-lib/httr2.

See Also

tabscreen_gpt(), tabscreen_gpt.tools()

Examples

## Not run: 
prompt <- "Is this study about a Functional Family Therapy (FFT) intervention?"

obj_with_error <-
  tabscreen_gpt(
    data = filges2015_dat[1:10,],
    prompt = prompt,
    studyid = studyid,
    title = title,
    abstract = abstract,
    model = "gpt-4o"
    )

obj_rescreened <-
  obj_with_error |>
  screen_errors()


## End(Not run)

Creating a temporary R environment API key variable

Description

This function automatically sets/creates an interim R environment variable with the API key to call a given AI model (e.g. ChatGPT). Thereby users avoid exposing their API keys. If the API key is set in the console, it will/can be revealed via the .Rhistory. Find more information about this issue at https://httr2.r-lib.org/articles/wrapping-apis.html.

Usage

set_api_key(key, env_var = "OPENAI_API_KEY")

Arguments

Details

When set_api_key() has successfully been executed, get_api_key() automatically retrieves the API key from the R environment and the users do not need to specify the API when running functions from the package that call the API. The API key can be permanently set by using usethis::edit_r_environ(). Then write ⁠OPENAI_API_KEY=[insert your api key here]⁠ and close the .Renviron window and restart R.

Value

A temporary environment variable with the name from env_var. If key is missing, it returns a password box in which the true API key can be entered.

Note

Find your personal API key via the OpenAI quickstart guide at https://developers.openai.com/api/docs/quickstart#generate-an-api-key.

See Also

get_api_key

Examples

## Not run: 
set_api_key()

## End(Not run)

Title and abstract screening with GPT API models using function calls via the original function call arguments

Description

This function has been deprecated (but can still be used) because OpenAI has deprecated the function_call and and functions argument which is used in this function. Instead use the tabscreen_gpt.tools() that handles the function calling via the tools and tool_choice arguments.

This function supports the conduct of title and abstract screening with GPT API models in R. This function only works with GPT-4, more specifically gpt-4-0613. To draw on other models, use tabscreen_gpt.tools(). The function allows to run title and abstract screening across multiple prompts and with repeated questions to check for consistency across answers. This function draws on the newly developed function calling to better steer the output of the responses. This function was used in Vembye, Christensen, Mølgaard, and Schytt. (2025).

Usage

tabscreen_gpt.original(
  data,
  prompt,
  studyid,
  title,
  abstract,
  ...,
  model = "gpt-4",
  role = "user",
  functions = incl_function_simple,
  function_call_name = list(name = "inclusion_decision_simple"),
  top_p = 1,
  time_info = TRUE,
  token_info = TRUE,
  api_key = get_api_key(),
  max_tries = 16,
  max_seconds = NULL,
  is_transient = gpt_is_transient,
  backoff = NULL,
  after = NULL,
  rpm = 10000,
  reps = 1,
  seed_par = NULL,
  progress = TRUE,
  messages = TRUE,
  incl_cutoff_upper = 0.5,
  incl_cutoff_lower = incl_cutoff_upper - 0.1,
  force = FALSE
)

Arguments

Value

An object of class "chatgpt". The object is a list containing the following components:

Note

The answer_data_sum data contains the following mandatory variables:

The answer_data_all data contains the following mandatory variables:

If any requests failed to reach the server, the chatgpt object contains an error data set (error_data) having the same variables as answer_data_all but with failed request references only.

The price_data data contains the following variables:

Find current token pricing at https://developers.openai.com/api/docs/pricing.

References

Vembye, M. H., Christensen, J., Mølgaard, A. B., & Schytt, F. L. W. (2025) GPT API Models Can Function as Highly Reliable Second Screeners of Titles and Abstracts in Systematic Reviews: A Proof of Concept and Common Guidelines https://psycnet.apa.org/record/2026-37236-001

Wickham H (2023). httr2: Perform HTTP Requests and Process the Responses. https://httr2.r-lib.org, https://github.com/r-lib/httr2.

Examples

## Not run: 

set_api_key()

prompt <- "Is this study about a Functional Family Therapy (FFT) intervention?"

tabscreen_gpt.original(
  data = filges2015_dat[1:2,],
  prompt = prompt,
  studyid = studyid,
  title = title,
  abstract = abstract,
  max_tries = 2
  )

 # Get detailed descriptions of the gpt decisions by using the
 # embedded function calling functions from the package. See example below.
 tabscreen_gpt.original(
   data = filges2015_dat[1:2,],
   prompt = prompt,
   studyid = studyid,
   title = title,
   abstract = abstract,
   functions = incl_function,
   function_call_name = list(name = "inclusion_decision"),
   max_tries = 2
 )

## End(Not run)

Title and abstract screening with GPT API models using function calls via the tools argument

Description

This function supports title and abstract screening using GPT API models in R. Specifically, it allows users to draw on all OpenAI GPT API completion models, including fine-tuned versions. The function enables title and abstract screening across multiple prompts, with repeated questions to assess consistency across responses. All of this can be performed in parallel. The function utilizes function calling, which is invoked via the tools argument in the request body. This is the main difference between tabscreen_gpt.tools() and tabscreen_gpt.original(). Function calls ensure more reliable and consistent responses to users' requests. See Vembye, Christensen, Mølgaard, and Schytt. (2025) for guidance on how adequately to conduct title and abstract screening with GPT models.

Usage

tabscreen_gpt.tools(data, prompt, studyid, title, abstract,
  api_url = "https://api.openai.com/v1/chat/completions", model = "gpt-4o-mini",
  role = "user", tools = NULL, tool_choice = NULL, top_p = 1,
  time_info = TRUE, token_info = TRUE, api_key = get_api_key(), max_tries = 16,
  max_seconds = NULL, is_transient = gpt_is_transient, backoff = NULL,
  after = NULL, rpm = 10000, reps = 1, seed_par = NULL, progress = TRUE,
  decision_description = FALSE, messages = TRUE, incl_cutoff_upper = NULL,
  incl_cutoff_lower = NULL, force = FALSE, custom_model = FALSE,
  fine_tuned = deprecated(), reasoning_effort = "medium", verbosity = "low",
  overinclusive = TRUE, ...)

tabscreen_gpt(data, prompt, studyid, title, abstract,
  api_url = "https://api.openai.com/v1/chat/completions", model = "gpt-4o-mini",
  role = "user", tools = NULL, tool_choice = NULL, top_p = 1,
  time_info = TRUE, token_info = TRUE, api_key = get_api_key(), max_tries = 16,
  max_seconds = NULL, is_transient = gpt_is_transient, backoff = NULL,
  after = NULL, rpm = 10000, reps = 1, seed_par = NULL, progress = TRUE,
  decision_description = FALSE, messages = TRUE, incl_cutoff_upper = NULL,
  incl_cutoff_lower = NULL, force = FALSE, custom_model = FALSE,
  fine_tuned = deprecated(), reasoning_effort = "medium", verbosity = "low",
  overinclusive = TRUE, ...)

Arguments

Value

An object of class 'gpt'. The object is a list containing the following datasets and components:

If the same question is requested multiple times, the object will also contain the following dataset with results aggregated across the iterated requests/questions.

Note

The answer_data data contains the following mandatory variables:

If any requests failed, the gpt object contains an error dataset (error_data) containing the same variables as answer_data but with failed request references only.

When the same question is requested multiple times, the answer_data_aggregated data contains the following mandatory variables:

The price_data data contains the following variables:

Find current token pricing at https://developers.openai.com/api/docs/pricing or model_prizes.

References

Vembye, M. H., Christensen, J., Mølgaard, A. B., & Schytt, F. L. W. (2025). Generative Pretrained Transformer Models Can Function as Highly Reliable Second Screeners of Titles and Abstracts in Systematic Reviews: A Proof of Concept and Common Guidelines. Psychological Methods. doi:10.1037/met0000769

Thomas, J. et al. (2024). Responsible AI in Evidence SynthEsis (RAISE): guidance and recommendations. https://osf.io/cn7x4

Wickham H (2023). httr2: Perform HTTP Requests and Process the Responses. https://httr2.r-lib.org, https://github.com/r-lib/httr2.

Examples

## Not run: 

library(future)

set_api_key()

prompt <- "Is this study about a Functional Family Therapy (FFT) intervention?"

plan(multisession)

tabscreen_gpt(
  data = filges2015_dat[1:2,],
  prompt = prompt,
  studyid = studyid,
  title = title,
  abstract = abstract
  )

plan(sequential)

 # Get detailed descriptions of the gpt decisions.

 plan(multisession)

 tabscreen_gpt(
   data = filges2015_dat[1:2,],
   prompt = prompt,
   studyid = studyid,
   title = title,
   abstract = abstract,
   decision_description = TRUE
 )

plan(sequential)


## End(Not run)

Title and abstract screening with GROQ API models using function calls via the tools argument

Description

This function supports the conduct of title and abstract screening with Groq API models in R. Specifically, it allows the user to draw on Groq-hosted models. The function allows to run title and abstract screening across multiple prompts and with repeated questions to check for consistency across answers. All of which can be done in parallel. The function draws on function calling which is called via the tools argument in the request body. Function calls ensure more reliable and consistent responses to ones requests. See Vembye, Christensen, Mølgaard, and Schytt. (2025) for guidance on how adequately to conduct title and abstract screening with GPT models.

Usage

tabscreen_groq(data, prompt, studyid, title, abstract, 
api_url = "https://api.groq.com/openai/v1/chat/completions",
 ..., model = "llama-3.1-8b-instant", role = "user", 
tools = NULL, tool_choice = NULL, top_p = 1, 
time_info = TRUE, token_info = TRUE, api_key = get_api_key_groq(), 
max_tries = 16, max_seconds = NULL, is_transient = .groq_is_transient, 
backoff = NULL, after = NULL, rpm = 10000, reps = 1, seed_par = NULL,
progress = TRUE, decision_description = FALSE, overinclusive = TRUE, 
messages = TRUE, incl_cutoff_upper = NULL, incl_cutoff_lower = NULL, 
force = FALSE)

Arguments

Value

An object of class "gpt". The object is a list containing the following components:

Note

The answer_data_aggregated data (only present when reps > 1) contains the following mandatory variables:

The answer_data data contains the following mandatory variables:

If any requests failed to reach the server, the object contains an error data set (error_data) having the same variables as answer_data but with failed request references only.

The price_data data contains the following variables:

Find current token pricing at https://groq.com/pricing.

References

Vembye, M. H., Christensen, J., Mølgaard, A. B., & Schytt, F. L. W. (2025). Generative Pretrained Transformer Models Can Function as Highly Reliable Second Screeners of Titles and Abstracts in Systematic Reviews: A Proof of Concept and Common Guidelines. Psychological Methods. doi:10.1037/met0000769

Thomas, J. et al. (2024). Responsible AI in Evidence SynthEsis (RAISE): guidance and recommendations. https://osf.io/cn7x4

Wickham H (2023). httr2: Perform HTTP Requests and Process the Responses. https://httr2.r-lib.org, https://github.com/r-lib/httr2.

Examples

## Not run: 

set_api_key_groq()

prompt <- "Is this study about a Functional Family Therapy (FFT) intervention?"

plan(multisession)

tabscreen_groq(
  data = filges2015_dat[1:2,],
  prompt = prompt,
  studyid = studyid,
  title = title,
  abstract = abstract,
  model = "llama3-70b-8192",
  max_tries = 2
  )
plan(sequential)

 # Get detailed descriptions of the decisions by using the
 # decision_description option.
plan(multisession)

 tabscreen_groq(
   data = filges2015_dat[1:2,],
   prompt = prompt,
   studyid = studyid,
   title = title,
   abstract = abstract,
   model = "llama3-70b-8192",
   decision_description = TRUE,
   max_tries = 2
 )
plan(sequential)

## End(Not run)

Title and abstract screening with OLLAMA API models using function calls via the tools argument

Description

This function supports the conduct of title and abstract screening with OLLAMA API models in R. Specifically, it allows the user to draw on locally hosted ollama models (e.g., Llama 3 / 3.1 variants, Mixtral/Mistral, Gemma, DeepSeek and Qwen). For more information on how to install and use OLLAMA, see https://docs.ollama.com/. Be aware that this function requires that you have OLLAMA installed and running on your local machine. The function allows to run title and abstract screening across multiple prompts and with repeated questions to check for consistency across answers. All of which can be done in parallel. The function draws on the newly developed function calling which is called via the tools argument in the request body. Function calls ensure more reliable and consistent responses to ones requests. See Vembye, Christensen, Mølgaard, and Schytt. (2025) for guidance on how adequately to conduct title and abstract screening with OLLAMA models.

Usage

tabscreen_ollama(data, prompt, studyid, title, abstract, 
api_url = "http://127.0.0.1:11434/api/chat", ..., model, role = "user", 
tools = NULL, tool_choice = NULL, top_p = 1, time_info = TRUE, 
max_tries = 16, max_seconds = NULL, backoff = NULL, after = NULL, 
reps = 1, seed_par = NULL, progress = TRUE, decision_description = FALSE, 
overinclusive = TRUE, messages = TRUE, incl_cutoff_upper = NULL, 
incl_cutoff_lower = NULL, force = FALSE)

Arguments

Value

An object of class "gpt". The object is a list containing the following components:

Note

The answer_data_aggregated data (only present when reps > 1) contains the following mandatory variables:

The answer_data data contains the following mandatory variables:

If any requests failed to reach the server, the object contains an error data set (error_data) having the same variables as answer_data but with failed request references only.

References

Vembye, M. H., Christensen, J., Mølgaard, A. B., & Schytt, F. L. W. (2025). Generative Pretrained Transformer Models Can Function as Highly Reliable Second Screeners of Titles and Abstracts in Systematic Reviews: A Proof of Concept and Common Guidelines. Psychological Methods. doi:10.1037/met0000769

Thomas, J. et al. (2024). Responsible AI in Evidence SynthEsis (RAISE): guidance and recommendations. https://osf.io/cn7x4

Wickham H (2023). httr2: Perform HTTP Requests and Process the Responses. https://httr2.r-lib.org, https://github.com/r-lib/httr2.

Examples

## Not run: 


prompt <- "Is this study about a Functional Family Therapy (FFT) intervention?"

plan(multisession)

tabscreen_ollama(
  data = filges2015_dat[1:2,],
  prompt = prompt,
  studyid = studyid,
  title = title,
  abstract = abstract,
  model = "llama3.2:latest",
  max_tries = 2
  )
plan(sequential)

 # Get detailed descriptions of the decisions by using the
 # decision_description option.
plan(multisession)

 tabscreen_ollama(
   data = filges2015_dat[1:2,],
   prompt = prompt,
   studyid = studyid,
   title = title,
   abstract = abstract,
   model = "llama3.2:latest",
   decision_description = TRUE,
   max_tries = 2
 )
plan(sequential)

## End(Not run)