convert.input.Rdconvert.input is a relatively generic function that applies the function fcn and inserts a record of it into the database. It is primarily designed for converting meteorological data between formats and can be used on observed data, forecasts, and ensembles of forecasts.
To minimize downloading and storing duplicate data, it first checks to see if a given file is already in the
database before applying fcn.
convert.input(input.id, outfolder, formatname, mimetype, site.id, start_date, end_date, pkg, fcn, con = con, host, browndog, write = TRUE, format.vars, overwrite = FALSE, exact.dates = FALSE, allow.conflicting.dates = TRUE, insert.new.file = FALSE, pattern = NULL, forecast = FALSE, ensemble = FALSE, ensemble_name = NULL, ...)
| input.id | The database id of the input file of the parent of the file being processed here. The parent will have the same data, but in a different format. |
|---|---|
| outfolder | The directory where files generated by functions called by convert.input will be placed |
| formatname | data product specific format name |
| mimetype | data product specific file format |
| site.id | The id of the site |
| start_date | Start date of the data being requested or processed |
| end_date | End date of the data being requested or processed |
| pkg | The package that the function being executed is in (as a string) |
| fcn | The function to be executed if records of the output file aren't found in the database. (as a string) |
| con | Database connection object |
| host | Named list identifying the machine where conversion should be performed.
Currently only |
| browndog | List of information related to browndog conversion. NULL if browndog is not to be used for conversion |
| write | Logical: Write new file records to the database? |
| format.vars | Passed on as arguments to |
| overwrite | Logical: If a file already exists, create a fresh copy? Passed along to fcn. |
| exact.dates | Ignore time-span appending and enforce exact start and end dates on the database input file? (logical) |
| allow.conflicting.dates | Should overlapping years ignore time-span appending and exist as separate input files? (logical) |
| insert.new.file | Logical: force creation of a new database record even if one already exists? |
| pattern | A regular expression, passed to |
| forecast | Logical: Is the data product a forecast? |
| ensemble | An integer representing the number of ensembles, or FALSE if it data product is not an ensemble. |
| ensemble_name | If convert.input is being called iteratively for each ensemble, ensemble_name contains the identifying name/number for that ensemble. |
| ... | Additional arguments, passed unchanged to |
A list of two BETY IDs (input.id, dbfile.id) identifying a pre-existing file if one was available, or a newly created file if not. Each id may be a vector of ids if the function is processing an entire ensemble at once.
convert.input executes the function fcn in package pkg via PEcAn.remote::remote.execute.R. All additional arguments passed to convert.input (...) are in turn passed along to fcn as arguments. In addition, several named arguments to convert.input are passed along to fcn. The command to execute fcn is built as a string.
There are two kinds of database records (in different tables) that represent a given data file in the file system. An input file contains information about the contents of the data file. A dbfile contains machine spacific information for a given input file, such as the file path. Because duplicates of data files for a given input can be on multiple different machines, there can be more than one dbfile for a given input file.
By default, convert.input tries to optimize the download of most data products by only downloading the years of data not present on the current machine. (For example, if files for 2004-2008 exist for a given data product exist on this machine and the user requests 2006-2010, the function will only download data for 2009 and 2010). In year-long data files, each year exists as a separate file. The database input file contains records of the bounds of the range stored by those years. The data optimization can be turned off by overriding the default values for exact.dates and allow.conflicting.dates.
If the flag forecast is TRUE, convert.input treats data as if it were forecast data. Forecast data do not undergo time span appending.
convert.input has the capability to handle ensembles of met data. If ensemble = an integer > 1, convert.input checks the database for records of all ensemble members, and calls fcn if at least one is missing. convert.input assumes that fcn will return records for all ensembles. convert.input can also be called iteratevely for each ensemble member. In this case ensemble_name contains the unique identifying name/number of the ensemble to be processed.