Skip to content
#

dask

Here are 193 public repositories matching this topic...

xarray
djhoese
djhoese commented Jul 30, 2019

There are various examples of how to define an AreaDefinition throughout the documentation and in test files like https://github.com/pytroll/pyresample/blob/master/pyresample/test/test_files/areas.yaml. However, some of these have projection definitions with only a defined (equatorial radius). This is technically incomplete and although it works in PROJ, rasterio/GDAL are a little more strict an

lesteve
lesteve commented May 19, 2020
from dask_jobqueue import SLURMCluster 
cluster = SLURMCluster(cores=1, memory='1GB') 
print(cluster.job_script()) 
#!/usr/bin/env bash

#SBATCH -J dask-worker
#SBATCH -n 1
#SBATCH --cpus-per-task=1
#SBATCH --mem=954M
#SBATCH -t 00:30:00

/home/lesteve/miniconda3/bin/python -m distributed.cli.dask_worker tcp://192.168.0.11:44065 --nthreads 1 --memory-limit 1000.00MB -
lr4d
lr4d commented Oct 8, 2020

Problem description

Our dask update graphs are not properly optimized.

We ussually use dask.dataframe optimization and set ave_width=repartition_ratio for kartothek.io.dask.dataframe.update_dataset_from_ddf graphs. We should return an optimized graph from update_dataset_from_ddf to make our users' life simple.

We already have code that does this, whoever picks this up can ping me

zblz
zblz commented Aug 15, 2017

Currently all of the metrics computed are independent of a target variable or column, but if lens.summarise took the name of a column as the target variable, the output of some metrics could be more interpretable even if the target variable is not used in any kind of predictive modelling.

A good example of this could be PCA (see #14), which could plot the different categories of the target va

nils-braun
nils-braun commented Oct 28, 2020

As the server speaks the presto protocol, which is understood by most of the BI tools, it would be nice to test and showcase it with some of them.
For example I have done a very quick test with both hue and metabase, which look promising - but also have shown some additional quirks in the server implementation.

Improve this page

Add a description, image, and links to the dask topic page so that developers can more easily learn about it.

Curate this topic

Add this topic to your repo

To associate your repository with the dask topic, visit your repo's landing page and select "manage topics."

Learn more

You can’t perform that action at this time.