rapids

For pandas API compatibility, we can implement Series.autocorr. autocorr calculates the Pearson correlation between the Series and itself lagged by N steps. Conceptually, this is a combination of shift and corr.

import pandas as pd
​
s = pd.Series([0.25, 0.5, 0.2, -0.05])
print(s.autocorr())
print(s

We no longer need to control the number of concurrent kernels, since now we control the number of concurrent tasks

libcudf AST supports timestamp types, and Spark's DateType is treated as a timestamp type in libcudf. We should be able to extend the existing AST expression support to include DateType inputs.

Let's show some examples of integration with kgextension
https://kgextension.readthedocs.io/en/latest/

Could be another notebook added to the tutorial.

Where it fits, we might also integrate as a dependency?

It would be nice to be able to set the initial_pool_size with a string like "500mb" or "2gb" as opposed to integer sizes like 500000000. We could vendor the code Dask uses to accomplish this:

https://github.com/dask/dask/blob/31af7f7040643c447a72c87a8f12457094ec15ff/dask/utils.py#L1171

In trying to write tests for #189, I'm finding very difficult to add columns to existing tests, as in some cases like the all_types table, the table is defined in a separate file than the tests and multiple tests try to write to the same table.

Additionally, our test suite doesn't prove that the data that are uploaded are the same as the data downloaded for all types.

We should consider m

As seen with gumdropsteve/turbo-telegram@3e2f3b3, the data for the first half of 2016 can be downloaded & preprocessed just like that of 2015. Is there any other data in the effective range? I.e. is pre-2015 data recorded the same?

If so, let's add it.