haver.direct {Haver} | R Documentation

## Set or Query the DLX Direct Mode

### Description

`haver.direct` sets and queries the DLX Direct mode. If the DLX Direct mode is switched on, databases access is directed to a Haver server. Otherwise local databases are accessed.

### Usage

    haver.direct()
    haver.direct(mode)

### Arguments

`mode` | a character or numeric 0/1 or logical `TRUE`/`FALSE` scalar. Specifies whether to switch DLX Direct on or off. `mode` must be one of 'off'/'on', 0/1, or FALSE/TRUE. In addition, 'force' is allowed (see below).

### Value

When no argument is supplied, a bool that holds the current DLX Direct setting; and `NULL` otherwise.

### Authentication

Upon package load, DLX Direct is always turned off. That is, the DLX Direct mode is not preserved between R sessions. Consequently, if you want to use DLX Direct, you have to issue `Haver.direct('on')` after each restart of the R kernel. At this point, two-factor authentication may be invoked, in which case a DLX window will open up, asking you for your email address and password. Then a security code will be sent to your email address. After entering the security code along with your password again, you are logged in.

Two-factor authentication will only be invoked again after a certain time interval. If you quit and restart R before two-factor authentication has expired, a call to `Haver.direct` requires no re-entry of credentials. An important implication of this is that there are barely any restrictions for scripting and automating tasks. If you have questions concerning details of the authentication process, please contact Haver support.

Once you are logged on to DLX Direct in an R session, you can turn DLX Direct mode on and off, instantly, and as often as you please.

Important: If your query wholly fails due to database access errors and you have verified that your query specification should work, try issuing `Haver.direct('force')`.

You can determine whether your query wholly failed due to database access by either looking at the text of an error message thrown by the Haver package or by verifying that a returned `HaverErrorReport` object has all entries in the ‘database.access’ code list. While this is frequently due to query misspecification, for example, because of a misspelled database name, it may happen in rare cases that the cause is a lost connection to the Haver server. `Haver.direct('force')` will then try to re-establish it. If the attempt fails, it will tell you so. In this case, you must restart your R console and re-issue `Haver.direct(1)` before you can successfully execute your query.

### Equivalence of Local and DLX Direct Queries

There are barely any restrictions on the metadata and data queries that you can execute with DLX Direct mode switched on. Almost any query that executes using local databases will also execute as a DLX Direct query.

### DLX Direct Query Speed and Interrupting Execution

The equivalence of local and DLX Direct queries is, naturally, subject to the caveat that queries over the internet take much longer than queries that use the local network. For example, querying the metadata of the local USECON database may take five seconds. That very same query, when run under `Haver.direct('on')`, may take two minutes (the exact timings in your setting may deviate from these numbers, of course). Therefore, if you have large-scale tasks in place that run on local databases, you may encounter difficulties executing these same tasks under DLX Direct mode, simply because the execution time is much longer.

If a DLX Direct query takes long, you can interrupt execution at any point and in the usual way (ESC key), so you will never be stuck with a running query. Nevertheless, we recommend the following:

- When you first start out doing DLX Direct queries, start small, see how long the queries take, and gradually work your way to larger queries. You can look at this command's examples section, which lists queries that grow in size.

- Whenever you do larger DLX Direct queries, think for a second about the time constraints imposed by the remote processing, and whether the expected execution time is acceptable to you.

### Differences Between Local and DLX Direct Queries

Query speed aside, there are very few and only minor exceptions to the equivalence of local and DLX Direct functionality. They are listed below.

- `Haver.databases` errors out if it is called under DLX Direct mode.

- Differences in query results arise if local databases are not up to date. In rare cases, differences may additionally occur in terms of `NA` paddings at the start and/or end of returned `HaverData` objects, even if local databases have been fully updated.

- Query limits, whether set by `haver.limits` or default ones, only apply to local queries, for the following reason: With remote queries, establishing whether a limit has been broken can take a considerable amount of time. Oftentimes it would be counterproductive to stop execution at that point.

### See Also

See Haver_package for all help entries related to the Haver package.

### Examples

    ## While example sections of other Haver commands are based on queries to example
    ## databases that ship with the Haver R package, the examples for 'haver.direct'
    ## below are cast in terms of live Haver databases. Your institution's
    ## subscription must include these databases for the examples to work. If one or
    ## several example statements use databases that you cannot access, you can try to
    ## rewrite these statements in terms of databases that are available to you.

    ## Before using DLX Direct, you must switch it on
    haver.direct(1)

    ## Our first DLX Direct query concerns the metadata from a small database, WBPRICES.
    hmd <- haver.metadata(database='wbprices')
    print( hmd )

    ## You can switch DLX Direct off, do local queries, and instantly turn DLX Direct mode
    ## back on.
    haver.direct(0)
    haver.path('examples')
    print( tail(haver.data('havermqa:gdp')) )
    haver.path('restore')

    haver.direct(1)
    print( tail(haver.data('usecon:gdp')) )

    ## You can use query functions without any restrictions on the function parameters;
    ## for example:
    ex_codes <- c('usecon:gdp', 'usecon:lanagra', 'eudata:s132gdpn', 'eudata:s134gdpn')
    hd <- haver.data(ex_codes, start=as.Date('1980-01-01'), frequency='A')
    hmd <- haver.datamd(hd)
    print( tail(hd) )

    ## Since DLX queries sometimes take a little time to stop when you try to cancel them (ESC key), it
    ## is important that you can assess their expected processing time. Below,
    ## we provide example queries that that will successively take longer.
    ## By executing them, you can quickly get a feel for the speed of DLX Direct queries
    ## in your particular setting.
    ## In the examples below, note that the rough number of codes queried and the rough expected execution time
    ## are as of August 2022. The expected execution time is dependent on your particular
    ## network setting and may therefore deviate considerably from the numbers given.

    ## Task: Getting lists of codes of an entire databases
    tt <- system.time
    print( tt(haver.dbcodes(database='wbprices')) ) ##     190 codes,     1 sec
    print( tt(haver.dbcodes(database='usecon')  ) ) ##  64,000 codes,   3-5 sec
    print( tt(haver.dbcodes(database='cmdty')   ) ) ## 300,000 codes, 15-20 sec

    ## Task: Getting metadata of an entire database
    print( tt(haver.metadata(database='wbprices')) ) ##    200 codes,   2-4 sec
    print( tt(haver.metadata(database='fdic')    ) ) ##  3,500 codes,  5-10 sec
    print( tt(haver.metadata(database='g10')     ) ) ## 19,000 codes, 20-30 sec
    print( tt(haver.metadata(database='usecon')  ) ) ## 64,000 codes,   1-3 min

    ## Task: Querying data from a single database
    ex_codes_g10  <- haver.dbcodes('g10'   , format='full')
    print( tt(haver.data(ex_codes_g10[1:10]  , frequency='A')) ) ##  1-3 sec
    print( tt(haver.data(ex_codes_g10[1:100] , frequency='A')) ) ##  1-3 sec
    print( tt(haver.data(ex_codes_g10[1:1000], frequency='A')) ) ##  4-6 sec
    print( tt(haver.data(ex_codes_g10[1:5000], frequency='A')) ) ## 8-12 sec

    ## As an aside to the above, note that 'haver.dbcodes' is the quickest way of
    ## determining the size of a database.
    print( tt(haver.dbcodes('g10')) )
    print( length(ex_codes_g10) )

    ## Task: Querying data from multiple databases.
    ## This is somewhat slower than single-database queries.
    ## We first shuffle codes from three different databases in order to get an input
    ## code list.
    ex_codes_eme  <- haver.dbcodes('emerge', format='full')
    ex_codes_pop  <- haver.dbcodes('unpop' , format='full')
    ex_codes_mix  <- c(ex_codes_g10[1:2000], ex_codes_eme[1:2000], ex_codes_pop[1:2000])
    set.seed(123456)
    ex_codes_mix <- sample(ex_codes_mix)
    print( ex_codes_mix[1:10] )

    print( tt(haver.data(ex_codes_mix[1:10]  , frequency='A')) ) ##   5-7 sec
    print( tt(haver.data(ex_codes_mix[1:100] , frequency='A')) ) ##   5-7 sec
    print( tt(haver.data(ex_codes_mix[1:1000], frequency='A')) ) ##   6-8 sec
    print( tt(haver.data(ex_codes_mix[1:5000], frequency='A')) ) ## 20-30 sec

    ## Task: Querying high-frequency (daily) data.
    ## Querying data of a higher frequency also may take a bit of extra time. Moreover,
    ## for series of daily frequency we must use
    ## 'haver.metadata' to get a proper database series list that excludes
    ## historical component series (see '?haver.metadata').
    ## The next few statements will construct our input code list.

    print( tt( hmd_dly <- haver.metadata(database='daily', histcodes='drop') ))  ## 30-60 sec
    ex_codes_dly <- paste(hmd_dly[['database']], hmd_dly[['code']], sep=':')
    print( length(ex_codes_dly) )

    ## We are now ready to time queries of daily series
    print( tt(haver.data(ex_codes_dly[1:10]  , frequency='D')) ) ##   1-3 sec
    print( tt(haver.data(ex_codes_dly[1:100] , frequency='D')) ) ##   1-3 sec
    print( tt(haver.data(ex_codes_dly[1:1000], frequency='D')) ) ##  8-12 sec

    ## WARNING: We are now going to query 5000 series of daily data.
    ## This is very often demanding in terms of memory requirements.
    ## It is likely to require several gigabyte of RAM. Only proceed if your system can handle this.
    ## Note that for local queries, you are better protected against running into
    ## queries that overburden your system via 'haver.limits'. You do not have this
    ## protection for DLX Direct queries.
    print( tt(haver.data(ex_codes_dly[1:5000], frequency='D')) ) ##   1-2 min

    ## If you would like to go back to local queries, simply switch DLX Direct off again.

    haver.direct(0)

[Package Haver version 3.2 Index]