readRawHeader() for TabularTextFile would produce an obscure “Error in if (!isEmpty) { : argument is of length zero” if the file is empty. Now it detects when the file is empty and gives a more informative error message.dsApplyInPairs() is defunct. Use future.apply::future_mapply() instead.extract() from the R.rsp package.dsApply(), which has been defunct since version 2.13.0 (April 2019). Use future.apply::future_lapply() instead.dsApplyInPairs() is deprecated. Use future.apply::future_mapply() instead.
dsApply() is defunct. Use future.apply::future_lapply() instead.
Removed defunct and hidden argument colClassPatterns of readDataFrame() for TabularTextFile. Use argument colClasses instead.
Removed defunct and hidden argument files of extractMatrix() for GenericTabularFileSet. Use extractMatrix(ds[files], ...) instead.
loadToEnv() for RDataFile was not declared an S3 method.Package requires R (>= 3.2.0) released April 2015.
Package no longer requires Bioconductor.
dsApply() is now deprecated. Instead, use future.apply::future_lapply().
dsApply() with .parallel = "BiocParallel::BatchJobs" and "BatchJobs" are now defunct. Instead, use future.apply::future_lapply() with one of the many backends that implements the Future API.
Now getChecksum() for ChecksumFile defaults to not creating a checksum file (which is the default for other types of file), but instead always return the checksum of the file by only calculating and in memory. This prevents for instance the equals() test on two different checksum files to generate another set of checksum files on themselves.
Now findByName() for GenericDataFileSet reports on the non-existing root paths in error messages.
GenericDataFile and GenericDataFileSet no longer report on memory (RAM) usage of objects.
dsApply(..., .parallel = "future") now used future_lapply() of the future package internally. dsApply() will soon be deprecated (see below).Argument colClassPatterns of readDataFrame() for TabularTextFile is now defunct. Use colClasses instead.
Argument files of extractMatrix() for GenericTabularFileSet is defunct.
dsApply() with either .parallel = "BiocParallel::BatchJobs" or "BatchJobs" is deprecated. Instead, use future::future_lapply() with whatever choice of future::plan() preferred.
Defunct argument aliased of getDefaultFullName() for GenericDataFile and defunct argument alias of GenericDataFileSet() have been removed.
Now file sizes are reported using IEC binary prefixes, i.e. bytes, KiB, MiB, GiB, TiB, …, YiB.
Added hasChecksumFile() for GenericDataFile.
hasBeenModified() for GenericDataFile gained argument update.
na.omit() for GenericDataFileSet; the default one in the stats package works equally well.Arguments$getTags() failed to drop missing values.
equals(df, other) for GenericDataFile would give an error if other was not a GenericDataFile.
dropTags() would drop name if a tag had the same name.
getOneFile() on a GenericDataFileSet with a single missing file would give an error, now it gives a file with an NA pathname.
Preparing to make the default pathname for GenericDataFile() to become NA_character_. It is currently NULL, but the goal is to enforce length(pathname) to be one.
extractMatrix(ds, files, ...) for GenericTabularFileSet is deprecated. Use extractMatrix(ds[files], ...) instead.
dsApply(..., .parallel = "future"), which utilizes the future package.Added support for sortBy(..., by = "mixedroman") for GenericDataFileSet.
Now commentChar = "" and commentChar = FALSE also disables searching for comment characters (just as commentChar = NULL) for TabularTextFile.
readDataFrame() for TabularTextFile with column-names translators set, could give an error “Number of read data columns does not match the number of column headers: …”. This was due to an update in utils::read.table() as of R v3.2.1 svn rev 68831.lapply(), dsApply() returns a list with names corresponds to the full names of the data set.getFullNames(..., onRemapping) to GenericDataFileSet to warn/err on full-name translations that generates inconsistent fullname-to-index maps before and after.linkTo(..., skip = TRUE) would give error “No permission to modify existing file: …” also in the case when the proper link already exists and there is no need to create a new one.
Now getReadArguments() for TabularTextFile let duplicated named colClasses entries override earlier ones, e.g. colClasses=c("*" = NA, "*" = "NULL", a = "integer") is effectively the same as colClasses=c("*" = "NULL", a = "integer"). Added package test.
nchar(..., type = "chars") is used internally for all file and directory names (including tags).as.character() for GenericDataFile with a missing (NA) pathname on recent R-devel (>= 2015-04-23) related to an update on how nchar() handles missing values.Now [[ for GenericDataFileSet returns a GenericDataFile not only if a numeric index is given but also if a character string is given.
Now argument idx for getFile() for GenericDataFileSet can also be a character string, in which case the file returned is identified using indexOf(..., pattern = idx, by = "exact", onMissing = "error").
Added RDataFile and RDataFileSet classes for *.RData files.
requireNamespace() instead of require() internally.as.character() for ChecksumFile gave an error when the checksum files was missing.Added support for sortBy(..., by = "filesize") and sortBy(..., decreasing = TRUE) for GenericDataFileSet.
Added rep() for GenericDataFileSet.
NOTES:
readDataFrame() would ignore argument colClasses iff it had no names. Added package system test for this case.commentChar = NULL for TabularTextFile:s failed.readChecksums() for ChecksumFileSet.byPath() for GenericDataFileSet would output verbose message enumerating files loaded to stdout instead of stderr.dsApply() for GenericDataFileSet would coerce argument verbose to logical before applying the function.sep for readDataFrame() would only work for , and \t; now it works for any separator.Now indexOf() first searched by exact names, then as before, i.e. by regular expression and fixed pattern matching. Added package system tests that contains particularly complicated cases for this. This was triggered by a rare but real use case causing an error in aroma.affymetrix. Thanks Benilton Carvalho for reporting on this.
Added argument by to indexOf() for GenericDataFileSet|List.
Added SuggestsNote field to DESCRIPTION with list of packages that are recommended for the most common use cases.
Bumped package dependencies.
ds[[idx]] instead of getFile(ds, idx) where possible.dsApply(..., .parallel = "none") would lower the verbose threshold before applying the function resulting is less verbose output in the non-parallel case.GenericDataFile would fail with linkTo() on Windows systems without necessary privileges. Made the test less conservative. Also, added an Rd section on privileges required on Windows for linkTo() to work. Thanks to Brian Ripley for reporting on this.NOTES:
readColumns() for TabularTextFile handles also header-less files.copyTo() for GenericDataFileSet no longer passes ... to byPath() when constructing the return data set.renameTo() passes ... to R.utils::renameFile() making it possible to also overwrite existing files.Added is.na() for GenericDataFile and GenericDataFileSet and na.omit() for the latter, which already supports anyNA().
Added linkTo() for GenericDataFile, which create a symbolic link at a given destination pathname analogously to how copyTo() creates a file copy at a given destination pathname.
copyTo() for GenericDataFile passes ... to R.utils::copyFile().copyTo() and renameTo() for GenericDataFile had verbose output enabled by default.digest2() is now defunct.Added duplicated(), anyDuplicated(), and unique() for GenericDataSet, which all compare GenericDataFile:s using the equals() method.
Now c() for GenericDataFileSet also works to append GenericDataFile:s. Added package system test for common use cases of c().
Added nbrOfColumns() for GenericTabularFile, which, if the number of columns cannot be inferred from the column names, will fall back to read the first row of data and use that as the number of columns.
Now nbrOfColumns() for ColumnNamesInterface returns NA if column names cannot be inferred and hence not be counted.
Now readDataFrame(..., header = FALSE) works as expected for tabular text files without headers.
Now getReadArguments() for TabularTextFile returns a colClasses vector of the correct length also in the case when there are no column names.
loadRDS() available for plain files and RdsFile:s.RdsFile and RdsFileSet objects for handling *.rds file sets.GenericSummary.ChecksumFile and ChecksumFileSet.extract() for GenericDataFileSet also handles when the data set to be extracted is empty, e.g. extract(GenericDataFileSet(), NA_integer_). Also, added support for argument onMissing = "dropall", which drops all files if one or more missing files where requested. Added package system tests for these case.GenericDataFileSet$byPath(..., recursive = TRUE) would be very slow setting up the individual files, especially for large data sets. Now it’s only slow for the first file.Added "[["(x, i) for GenericDataFileSet, which gets a GenericDataFile by index i in [1,length(x)]. When i is non-numeric, the next "[["(x, i) method in the class hierarchy is used, e.g. the one for Object:s.
Added gzip()/gunzip() for GenericDataFileSet.
Added anyNA() to GenericDataFileSet to test whether any of the pathnames are NA, or not.
getChecksum() on GenericDataFile:s and GenericDataFileSet:s.append() to become a generic function does now call base::append() in the default, instead of copy the latter. All this will eventually be removed, when proper support for c, [, [[, etc. has been added everywhere.getChecksum() from R.cache instead of creating its own. This solves the problem of the default getChecksum() of R.cache not being found.readDataFrame() for TabularTextFile subsets by row, before reparsing numerical columns that were quoted.autoload():s used internally.Deprecated digest2() and deprecated -> defunct -> dropped.
Now GenericDataFileSet() gives an error informing that argument alias is defunct.
Now no generic functions are created for defunct methods.
R.filesets Package object is also available when the package is only loaded (but not attached).cat() from R.utils.SPEEDUP: Package no longer uses R.utils::whichVector(), which use to be 10x faster, but since R 2.11.0 which() is 3x times again.
Package no longer utilizes import(), only importFrom():s.
WORKAROUND: For now, package attaches the R.oo package. This is needed due to what appears to be a bug in how R.oo finalizes Object:s assuming R.oo is/can be attached. Until that is resolved, we make sure R.oo is attached.
Forgot to import R.methodsS3::appendVarArgs().
[() and c() for GenericDataFileSet.private = FALSE to byPath() of GenericDataFileSet.isGzipped() ignores the case of the filename extension when testing whether the file is gzipped or not.rm() calls with NULL assignments.digest2(), which soon will be deprecated.\usage{} lines are at most 90 characters long.In addition to a fixed integer, argument skip for readDataFrame() (default and for TabularTextFile) may also specify a regular expression matching the first row of the data section.
Now argument skip to TabularTextFile and readDataFrame() for that class causes the parser to skip that many lines including commented lines, whereas before it did not count commented lines.
Added a default readDataFrame() for reading data from one or more tabular text files via the TabularTextFile/TabularTextFileSet classes.
colClassPatterns of readDataFrame() for TabularTextFile has been renamed to colClasses.startupMessage() of R.oo.indexOf() for GenericDataFileSet throws an exception if user tries to pass an argument names.Added head() and tail() for GenericTabularFile.
Added subsetting via [() to GenericTabularFile.
nbrOfRows() for TabularTextFile forgot to exclude comment rows in the file header.
readColumns() for GenericTabularFile would not preserve the order of the requested columns.
Added getOneFile() for GenericDataFileSet, which returns the first GenericDataFile with a non-missing pathname.
Added argument absolute = FALSE to getPathname() for GenericDataFile.
GenericDataFile stores the absolute pathname of the file, even if a relative pathname is given. This makes sure that the file is found also when the working directory is changed.equals() for GenericDataFileSet would only compare the first GenericDataFile in each set.isGzipped() to GenericDataFile.writeColumnsToFiles() to GenericTabularFile. Used to be available only for TabularTextFile.getDefaultColumnNames() for TabularTextFile did not use columnNames if it was set when creating the TabularTextFile object.
Now getReadArguments() for TabularTextFile drops arguments that are NULL, because they could cause errors downstreams, e.g. readDataFrame() calling read.table(..., colClasses = NULL) => rep_len(NULL, x) => “Error in rep_len(colClasses, cols) : cannot replicate NULL to a non-zero length”.
as.list() for GenericDataSet to return a named list of GenericDataFile:s (previously it had no names). The names are the (translated) full names of the GenericDataFile:s.lapply() and sapply() for GenericDataSet, because the corresponding functions in the base package utilizes as.list().Now GenericDataFile() retrieves the file time stamps such that hasBeenModified() returns a correct value also when first called, and not only TRUE just in case. This has the effect that getChecksum() will detected cached results already at the second call as long as the file has to been modified. Previously it took two calls to getChecksum() for it to be properly cached.
Now declaring more internal and temporary Object fields as “cached”, which means they will be cleared if clearCache() or gc() is called on the corresponding object.
Added further verbose output to TabularTextFileSet.
DOCUMENTATION: Minor corrections to help pages.
NOTES:
TabularTextFile to ignore header comment arguments when inferring column names and classes.clearCache() for GenericDataFileSet relies on ditto of Object to clear all cached fields (= with field modifier "cached").{get,set}Label() for GenericDataFile and {get,set}Alias() for GenericData{File,FileSet}. Related arguments such at alias to GenericDataFileSet and aliased to getDefaultFullName() for GenericDataFile are also deprecated.seq_along(x) instead of seq(along = x) everywhere. Similarly, seq(ds) where ds is GenericDataFileSet is now replaced by seq_along(ds). Likewise, seq_len(x) replaces seq(length = x), and length(ds) replaces nbrOfFiles(ds).Now TabularTextFile() tries to infer whether the data section contains column names or not. This is done by comparing to the optional columnNames header argument. If that is not available, it will (as before) assume there are column names.
Now readDataFrame() acknowledge header comment arguments columnNames and columnClasses if specified in the file.
Now getDefaultColumnNames() for TabularTextFile falls back to header comment argument columnNames, if there are no column names in the actual data table.
Now readRawHeader() for TabularTextFile also parses and returns header comment arguments.
ColumnNamesInterface which GenericTabularFile now implements. Classes inheriting from GenericTabularFile should rename any getColumnNames() method to getDefaultColumnNames().whichVector() with which(), because the latter is now the fastest again.setColumnNames() for GenericTabularFile, which utilizes setColumnNamesTranslator().{get,set}ColumnNameTranslator() in favor of {get,set}ColumnNamesTranslator(); note the plural form.readDataFrame() for TabularTextFile no longer returns attribute fileHeader, unless argument debug is TRUE.validate() to GenericDataFileSet, which iteratively calls validate() on all the GenericDataFile:s in the set. The default is to return NA, indicating that no validation was done.Arguments$getReadablePath() instead of filePath(..., expandLinks = "any").Arguments$getFilename() below.... to NextMethod(), cf. R-devel thread ‘Do not pass’…’ to NextMethod() - it’ll do it for you; missing documentation, a bug or just me?’ on Oct 16, 2012.Arguments$getFilename() from this package to R.utils v1.17.0.fromFiles() for GenericDataFileSet is now defunct in place for byName(), which has been recommended since January
Now readDataFrame() for TabularTextFile defaults to read strings as characters rather than as factors. To read strings as factors, just pass argument stringsAsFactors = TRUE.
Added readDataFrame() for TabularTextFileSet.
ROBUSTNESS: Now getHeader() for TabularTextFile checks if the file has been modified before returned cached results.
trim() being overridden by ditto from the IRanges package, iff loaded.extractMatrix() for GenericTabularFile adds column names just as ditto for GenericTabularFileSet does..Internal() calls.GenericDataFile and GenericDataFileSet handle so called “empty” files, which are files with NULL pathnames.getCommentChar() to TabularTextFile and argument commentChar to its constructor. This allows to use custom comment characters other than just "#".GenericDataFileSet$byName(..., subdirs) would throw Error in strsplit(subdirs, split = "/\\") iff subdirs != NULL.
Improved the handling of the newly introduced depth parameter, e.g. by making it optional/backward compatible.
GenericDataFileSet, such that one can correctly infer fullname and subdirs from the path.named to getTags() for FullNameInterface. If TRUE, tags of format "<name>=<value>" will be parsed and returned as a named "<value>", e.g. "foo,n=23,bar,n=42" is returned as c("foo", "n"="23", "bar", "n"="42").readDataFrame(..., colClasses = ..., trimQuotes = TRUE) of TabularTextFile will read numeric columns that are quoted. This is done by first reading them as quoted character strings, dropping the quotes, and then rereading them as numeric values..fileClass to appendFiles() for GenericDataFileSet.ROBUSTNESS: Now appendFiles() for GenericDataFileSet asserts that all files to be appended are instances of the file class of this set as given by the static getFileClass().
ROBUSTNESS: Added argument .assertSameClass to appendFiles() for GenericDataFileSet, which if TRUE asserts that the files to be appended inherits from the same class as the existing files. Before this test was mandatory.
getChecksum() to GenericDataFileSet, which calculates the checksum of the object returned by the protected getChecksumData(). Use with care, because what objects should be the basis of the checksum is not clear, e.g. should it be only the file system checksum, or should things such as translated fullnames be included as well?equals() for GenericDataFile would consider two files not to be equal only if their checksums was equal, and vice verse. Also, when creating the message string explaining why they differ an error would have been thrown.hpaste() internally wherever applicable.appendFullNameTranslatorBy<what>() for <character> and <function> assert that the translator correctly returns exactly one string. This has the effect that setFullName() and friends are also tested.Added = to the list of safe characters for Arguments$getFilename().
Added fullname(), name(), tags(), and dropTags().
findByName() for GenericDataFileSet it would throw “<simpleError in paths[sapply(rootPaths, FUN = isDirectory)]: invalid subscript type ‘list’>” in case no matching root path directories existed.Added dropRootPathTags().
GENERALIZATION: Added support to findByName() for GenericDataFileSet such that root paths also can be specified by simple regular expression (still via argument paths). Currently it is only the last subdirectory that can be expanded, e.g. foo/bar/data(,.*)/.
GENERALIZATION: Now byName() for GenericDataFileSet will try all possible data set directories located when trying to setup a data set. Before it only tried the first one located. This new approach is equally fast for the first data set directory as before. The advantage is that it adds further flexibilities, e.g. the first directory may not be what we want but the second, which can be further tested by the byPath() and downstream methods such as the constructor.
ROBUSTNESS: Now writeColumnsToFiles() for TabularTextFile writes files atomically, which should minimize the risk for generating incomplete files.
getTags() for Arguments from aroma.core package.fromFiles() of GenericDataFileSet has been deprecated, if still called by someone.GENERALIZATION: Now append() for GenericDataFileSet tries to also append non-GenericDataFileSet object by passing them down to appendFiles() assuming they are GenericDataFile:s.
GENERALIZATION: Now appendFiles() for GenericDataFileSet also accepts a single item. Thus, there is no longer a need to wrap up single items in a list.
ROBUSTNESS: Now GenericDataFileSet$byName() asserts that arguments name and tags contain only valid characters. This will for instance prevent passing paths or pathnames by mistake.
Now appendFullNameTranslator(..., df) for FullNameInterface takes either pattern or fixed translations in data.frame.
Added sortBy() to GenericDataFileSet, which sorts files either in a lexicographic or a mixedsort order.
DOCUMENTATION: Added more Rd help pages.
DOCUMENTATION: Removed any duplicated \usage{} statements from the Rd documentation.
indexOf() for GenericDataFileSet/List would return NA if the search pattern/string contained parentheses. The reason is that such have a special meaning in regular expression. Now indexOf() first search by regular expression patterns, then by fixed strings. Thanks Johan Staaf at Lund University and Larry(?) for reporting on this issue.Now GenericDataFileSet$findByName(..., mustExist = FALSE) do no longer throw an exception even if there is no existing root path.
Added argument firstOnly = TRUE to findByName() for GenericDataFileSet.
Added appendFullNameTranslatorBy...() methods to the FullNameInterface class for data frames, TabularTextFile:s, and TabularTextFileSet:s.
"NA" to the default na.strings returned by getReadArguments() for TabularTextFile.NOTES:
.onUnknownArgs to GenericDataFile() and GenericDataFileSet(). As before, the default is to throw an exception if there are unknown arguments. However, in certain case it is useful to allow (and ignore) “stray” arguments.indexOf() of GenericDataFileSet and GenericDataFileSetList did not handle names with regular expression symbols + and *. Thanks to Randy Gobbel for the initial error report.GenericDataFile and GenericDataFileSet.fromFiles() of GenericDataSet. Use byPath() instead.files is logical, then extract() of GenericDataFileSet and GenericDataFileSetList now asserts that the length of files matches the number of available files.exData/.readColumns(..., column=<string>) on a TabularTextFile would give “Error … object ‘columnNames’ not found”.default = "\\.([^.]+)$" to getExtensionPattern() of GenericDataFile. Before the default value was hard coded inside this function.setExtensionPattern(..., pattern = NULL) of GenericDataFile works.Added protected as.data.frame() to GenericDataFileSetList.
Now GenericDataFile(NA, mustExist = FALSE) is a valid object. Made all methods aware of such missing files.
Now extract(ds, c(1, 2, NA, 4), onMissing = "NA") returns a valid GenericDataFileSet where missing files are returned as missing GenericDataFile:s.
Added na.rm = TRUE to all getTags() so that it returns NULL in case the file is missing.
copyTo() of GenericDataFileSet quietly ignores missing files.
Added Rd help for indexOf() of GenericDataFileSet.
ROBUSTNESS: Using new Arguments$getInstanceOf() were possible.
Now all index arguments are validated correctly using the new max argument of Arguments$getIndices(). Before the case where max == 0 was not handled correctly.
Changed the default to parent = 0 for getDefaultFullName() of GenericDataFileSet to be consistent with the documentation.
Now GenericDataFile(pathname) throws an error if pathname is referring to a directory.
getPath() and getDefaultFullName() of GenericDataFileSet would return a logical instead of character value.
indexOf(ds, names) of GenericDataFileSet would return a logical instead of an integer vector of NA:s if none of the names existed.
translateFullName() of FullNameInterface and translateColumnNames() of GenericTabularFile throw an exception if some fullnames were translated into NA. They also assert that no names were dropped or added in the process.After doing append() to a GenericDataFileSet, the total file size reported would remain the same.
Appending empty data sets using append() of GenericDataFileSet would give error ‘Error in this$files[[1]] : subscript out of bounds’.
Added {get,set}ExtensionPattern() to FullNameInterface.
Added getExtension() to GenericDataFile.
appendFullNameTranslatorBylist() which makes it possible to do setup a sequence of fullnames translators fnt1, fnt2, fnt3 by calling setFullNameTranslator(..., list(fnt1, fnt2, fnt3)).Added support for having a sequence of fullname translator functions. These can be added using appendFullNameTranslator().
Added an example() to FullNameInterface.
[() to TabularTextFile.Added the FullNameInterface, which is the interface class that defines what fullnames, names, tags etc are.
Now setFullName*s*Translator() for GenericDataFileSet dispatches on the by argument. If that is not possible, it call setFullNameTranslator() for each file in the set (as before).
GenericDataFile and GenericDataFileSet implements the FullNameInterface, which mean less redundant code.fromFiles() to byPath(). For backward compatibility the former calls the latter.findByName() of GenericDataFileSet follows Windows Shortcut links also for subdirectories.Analogously to the method for a GenericDataFile, the setFullNameTranslator() method for GenericDataFileSet now assumes that the fullname translator function accepts also argument set.
Added argument .fileSetClass to GenericDataFileSet().
GenericDataFile should accept any number of arguments. The first argument will always be (an unnamed) argument containing the name (or names) to be translated. If the translator is for a GenericDataFile, an additional argument file will also be passed. This allows the translator function to for instance read the file header and infer the name that way.Extracted several classes and methods from the aroma.core package.
Created package.