spark
Here are 4,539 public repositories matching this topic...
-
Updated
Apr 16, 2020 - Go
Issue Description
The following test fails for seed = 0 but passes for (as far as I can tell) any other seed (e.g. seed = 1)
https://gist.github.com/orausch/9a42e24b782319447a515e8c29b364a0
Version Information
Please indicate relevant versions, including, if relevant:
- Deeplearning4j version:
beta6 - Platform information (OS, etc): Ubuntu 19.10
(cc @rpatra)
Environment:
- Framework: PyTorch
- Framework version: 1.3.1
- Horovod version: 0.19.0
- MPI version: 4.0.2
- CUDA version: N/A
- NCCL version: N/A
- Python version: 3.7.5
- OS and version: Mac OS 10.15.2
- GCC version: 9.2.0
Checklist:
- Did you search issues to find if somebody asked this question before? Yes
- If your question is about hang, did you read [this d
If I were to deploy cube.js using AWS serverless architecture, is athena required?
The docs https://cube.dev/docs/deployment#serverless do not mention whether Athena is optional or required. But when reading it, I assume it is required because there are athena keys in the serverless.yml config. I'm evaluating the idea of using Postgres RDS as the sole datasource for cubejs.
-
Updated
Oct 31, 2019
how to define java8 when submit application use spark-submit
name: Bug report/Feature request/Question
about: Create a report to help us improve
title: ''
label: bug/enhancement/question
assignees: ''
Environment:
- Java version:
- Scala version:
- Spark version:
- PyTorch and Python version:
- OS and version:
Checklist:
- Did you check if your bug/feature/
-
Updated
Apr 21, 2020 - Java
I refer to this teaching document and an error occurs
https://thingsboard.io/docs/user-guide/rule-engine-2-0/tutorials/rpc-reply-tutorial/
root@ubuntu:~# curl -X POST -d '{"method": "getTemperature", "params":{}}' http://localhost:8080/api/v1/0E2GoD82WeXkYYkwrRikent-Type:application/json"
**{"times
if I understood it corretly from README.MD, we can install like this:
$ git clone https://github.com/donnemartin/dev-setup.git && cd dev-setup
$ ./.dots bootstrap osxprep brew osx
and later when we need datastores, we run
$ cd ~/dev-setup
$ ./.dots datastores
I understand that bootstrap copies the dot files to the home directory, such as .bash_profile and .exports.
but
-
Updated
Apr 24, 2020 - Jupyter Notebook
Is your feature request related to a problem? Please describe.
no
Describe the solution you'd like
Alluxio add presto under catalog type that make parquet file which stored in Alluxio as a table just like Spark
Urgency
There is a big demand that users want to do some ad-hoc on presto with data produced by spark.
Now they need create table in hive first and sync metastore in
File "/root/miniconda3/bin/pipeline", line 11, in <module>
sys.exit(_main())
File "/root/miniconda3/lib/python3.7/site-packages/cli_pipeline/cli_pipeline.py", line 5734, in _main
_fire.Fire()
File "/root/miniconda3/lib/python3.7/site-packages/fire/core.py", line 127, in Fire
component_trace = _Fire(component, args, context, name)
Fil
-
Updated
Mar 16, 2020 - Python
Spark 2.3 officially support run on kubernetes. While our guide of "Run on Kubernetes" is still based on a special version of Spark 2.2, which is out of date. We need to:
- update that document to Spark 2.3
- release the corresponding docker images.
-
Updated
May 26, 2019 - Scala
According to the generated build
The commands to launch are the following :
docker pull andypetrella/spark-notebook:0.7.0-scala-2.11.8-spark-2.1.1-hadoop-2.7.2-with-hive
docker run -p 9001:9001 andypetrella/spark-notebook:0.7.0-scala-2.11.8-spark-2.1.1-hadoop-2.7.2-with-hive
Using that image (and I think it i
Hello,
I was able to run python scripts in dev mode using the steps provided in documentation. but for production, I am not sure which all folders to keep and the process to follow. editing the local conf and local sh files and running the server_deploy script, I was able to generate the server jar. But still i had to manually add the python context and upload my egg file.
Can someone pleas
I have noticed a small error in the documentation around S3 configurations:
https://docs.delta.io/latest/delta-storage.html#amazon-s3
On the read part, it should be load and not save:
spark.read.format("delta").load("s3a://<your-s3-bucket>/<path>/<to>/<delta-table>")
Also, I have successfully tested Delta 0.5.0 with on-premise S3 - https://min.io
There were some quirks around the
Use this checklist to track logstash wiki and documentation
- Update https://github.com/Cyb3rWard0g/HELK/wiki/Create-Plugins-Offline-Package
- Update HELK overview picture
- remove alien vault integration
- update with new indexes #256
- document catch all for windows
- document indexme catch all
- some general guideline on X amount of devices and or X amo
Series.reindex
Implement Series.reindex.
https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.Series.reindex.html
Version
com.microsoft.ml.spark:mmlspark_2.11:jar:0.18.1
spark= 2.4.3
scala=2.11.12
data (csv with header) https://gist.github.com/ttpro1995/69051647a256af912803c9a16040f43a
download data and save as csv file, put into folder /data/public/HIGGS/higgs.test.predictioncsv
val data = spark.read.option("header","true").option("inferSchema", "true").csv("/data/public/HIGGS
Problem
Some of our transformers & estimators are not thoroughly tested or not tested at all.
Solution
Use OpTransformerSpec and OpEstimatorSpec base test specs to provide tests for all existing transformers & estimators.
-
Updated
Aug 19, 2019 - R
Improve this page
Add a description, image, and links to the spark topic page so that developers can more easily learn about it.
Add this topic to your repo
To associate your repository with the spark topic, visit your repo's landing page and select "manage topics."
Alexnet implementation in tensorflow has incomplete architecture where 2 convolution neural layers are missing. This issue is in reference to the python notebook mentioned below.
https://github.com/donnemartin/data-science-ipython-notebooks/blob/master/deep-learning/tensor-flow-examples/notebooks/3_neural_networks/alexnet.ipynb