dask

I'm hoping to get an idea of the memory size of a dask.dataframe once I call .compute() on it

My current approach is

import dask.dataframe as dd
from dask.utils import format_bytes

ddf = dd.demo.make_timeseries(
    start="2000-01-01",
    end="2000-01-02",
    dtypes={"x": float, "y": float, "id": int},
    freq="10ms",
    partition_freq="24h",
)

format_bytes(ddf.memory_u

Is your feature request related to a problem? Please describe.
Sometimes you want to check that data values are present in another array, but only up to a certain tolerance.

Describe the solution you'd like
da.isin(test_values, tolerance=1e-6), where the tolerance argument is optional.

Not sure what the implementation should be but there are two vectorized [suggestions here](http

Support Series.median()

The stumpy.snippets feature is now completed in #283 which follows this work:

• The original published paper
• The supporting website with data examples

We have a rough notebook t

What happened:

When creating a LocalCluster object the comm is started on a random high port, even if there are no other clusters running.

What you expected to happen:

Should use port 8786.

Minimal Complete Verifiable Example:

$ conda create -n dask-lc-test -c conda-forge -y python=3.8 ipython dask distributed
$ conda activate dask-lc-test

The `d

Describe the bug
According to the multiscene documentation, the property all_same_area does:

Determine if all contained Scenes have the same ‘area’.

However, I have created a multiscene where all scenes have the same area (they just differ between datasets), yet the property returns Fa

Code Sample, a minimal, complete, and verifiable piece of code

from pyresample.boundary import Boundary
b = Boundary(my_lons, my_lats)
print(b.contour_poly.area())

Problem description

The above code doesn't fail if the provided lons/lats are 2D (not sure on 3D+), but the class and all functions/utilities underneath it assume 1D arrays. The end results are incor

The ML implementation is still a bit experimental - we can improve on this:

• SHOW MODELS and DESCRIBE MODEL
• Hyperparameter optimizations, AutoML-like behaviour
• @romainr brought up the idea of exporting models (#191, still missing: onnx - see discussion in the PR by @rajagurunath)
• and some more showcases and examples

from dask_jobqueue import SLURMCluster 
cluster = SLURMCluster(cores=1, memory='1GB') 
print(cluster.job_script()) 

#!/usr/bin/env bash

#SBATCH -J dask-worker
#SBATCH -n 1
#SBATCH --cpus-per-task=1
#SBATCH --mem=954M
#SBATCH -t 00:30:00

/home/lesteve/miniconda3/bin/python -m distributed.cli.dask_worker tcp://192.168.0.11:44065 --nthreads 1 --memory-limit 1000.00MB -

Problem description

Reading a dataset with eager's read functionality raises a ValueError when providing columns.

Example code (ideally copy-pastable)

import pandas as pd

from tempfile import TemporaryDirectory
from functools import partial
from storefact import get_store_from_url

from kartothek.io.eager import store_dataframes_as_dataset, read_dataset_as_data

Is your feature request related to a problem? Please describe.
asv performance now depends on local machine performance

Describe the solution you'd like
Independent of my laptop

https://labs.quansight.org/blog/2021/08/github-actions-benchmarks/

Describe alternatives you've considered
keep as is

Currently all of the metrics computed are independent of a target variable or column, but if lens.summarise took the name of a column as the target variable, the output of some metrics could be more interpretable even if the target variable is not used in any kind of predictive modelling.

A good example of this could be PCA (see #14), which could plot the different categories of the target va

In determining the correct reader for the file provided we currently have two options (as of #224).

1. Providing reader param to AICSImage (i.e. img = AICSImage("s3://some-file.ext", reader=readers.lif_reader.LifReader)
2. Not providing a reader, and AICSImage looping over all SUPPORTED_READERS.

Option 1 is the fastest + safest method for loading a file into AICSImage (without using

Without thinking I put resampling="bilinear" and got an error when I called .compute()

Traceback (most recent call last):
  File "carajas.py", line 92, in <module>
    band_medianNP = band_median.compute()
  File "/home/ubuntu/anaconda3/envs/richard/lib/python3.8/site-packages/xarray/core/dataarray.py", line 899, in compute
    return new.load(**kwargs)
  File "/home/ubuntu/anaco

As per @krfricke, we should create a single RayCommunication actor class that holds both an asyncio Queue and asyncio Event. This way we only need one remote actor for all communication between training workers and the driver, and we don't have to be dependent on ray.util.Queue