uber / petastorm

Watch 40
Star 859
Fork 154

Code
Issues 89
Pull requests 15
Actions
Projects 1
Wiki
Security
Insights

Code Issues 89 Pull requests 15 Projects 1 Actions Wiki Security Pulse

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Pick a username

Email Address

Password

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

89 Open 99 Closed

Author

Filter by author

Label

Filter by label

Use alt + click/return to exclude labels.

Projects

Filter by project

Milestones

Filter by milestone

Assignee

Filter by who’s assigned

Sort

Sort by

Newest Oldest Most commented Least commented Recently updated Least recently updated

Most reactions

Some errors happen with the code rows_rdd = rows_rdd.map(lambda x:dict_to_spark_row(schema,x))

#560 opened Jun 10, 2020 by cmh14

NdarrayCodec does not implement __str__

#559 opened Jun 4, 2020 by dmcguire81

walk method in GCSFSWrapper returns empty string as one of filenames

#558 opened Jun 3, 2020 by aleks-djuric

1 2

Spark Dataset Converter reset reader position does not work as expected

#553 opened May 8, 2020 by liangz1

Importing ABC directly from collections was deprecated and will be removed in Python 3.10. Use collections.abc

#550 opened Apr 26, 2020 by tirkarthi

global context not imported in transform_spec function with reader_pool_type="process"

#524 opened Mar 27, 2020 by kaiwenw

Petastorm PyTorch dataloader slower than JSON

#519 opened Mar 25, 2020 by kaiwenw

Petastorm sharding + Distributed PyTorch

#508 opened Mar 19, 2020 by megaserg

ast Syntax error when parsing non-petastorm dataset

#507 opened Mar 17, 2020 by working-estimate

Cannot resolve error "Cannot auto-create unischema due to unsupported column type"

#501 opened Mar 4, 2020 by muntasirraihan

Suspected memory leak with `reader_pool_type='process'`

#500 opened Mar 4, 2020 by megaserg

parquet.enable.summary-metadata and add_metadata are deprecated

#499 opened Mar 3, 2020 by filipski

Petastorm slow performance when parsing a large column dataset

#493 opened Feb 24, 2020 by jerrygb

Implement detailed logging for hdfs nameservice/namenode resolution

#489 opened Feb 14, 2020 by selitvin

Petastorm "float division by zero" when applying filter predicate on a dataset which is partitioned on more than one column

#487 opened Feb 13, 2020 by jamesprinc3

ImportError: cannot import objects

#483 opened Jan 31, 2020 by Vaibhav47Sharma

Verify that list-of-lists can be supported by Petastorm

#473 opened Jan 25, 2020 by selitvin

Is there support for gs datasets?

#468 opened Jan 13, 2020 by skeller88

dataset.make_one_shot_iterator() raises AttributeError: 'MapDataset' object has no attribute 'make_one_shot_iterator'

#467 opened Jan 13, 2020 by skeller88

Flaky test test_with_batch_reader[2] ?

#466 opened Jan 11, 2020 by selitvin

ArrowIOError: Corrupted file, smaller than file footer

#462 opened Dec 22, 2019 by balajib5497

field added by transformation is omitted if NGram is defined

#461 opened Dec 19, 2019 by GregAru

InvalidArgumentError: parquet array column not being transformed to dataset

#458 opened Dec 6, 2019 by DASpringate

"Error in sys.excepthook" on exit when using make_reader with hdfs_driver libhdfs in python 3.x

#455 opened Dec 5, 2019 by bhuntley

issue with Unischema to spark schema conversion

#451 opened Nov 27, 2019 by msaisumanth

Previous 1 2 3 4 Next

Previous Next

ProTip! What’s not been updated in a month: updated:<2020-05-18.