data-engineering
Here are 1,150 public repositories matching this topic...
-
Updated
Mar 7, 2022
-
Updated
Jan 2, 2022
-
Updated
Jan 25, 2022
Hello!
I've found an issue here:[Bitbucket Storage](https://docs.prefect.io/api/latest/storage.html#github) is a storage option that uploads flows to a Bitbucket repository as .py files.
Page reference: https://docs.prefect.io/orchestration/flow_config/storage.html#bitbucket
First, the link is incorrect. Second, should the line read more like Github where it references the repository
Describe the bug
data docs columns shrink to 1 character width with long query
To Reproduce
Steps to reproduce the behavior:
- make a batch from a long query string
- run validation
- render result to data docs
- See screenshot
<img width="1525" alt="Data_documentation_compiled_by_Great_Expectations" src="https://user-images.githubusercontent.com/928247/103230647-30eca500-4
We should do something like https://blog.questionable.services/article/kubernetes-deployments-configmap-change/ to ensure that if the pod sweeper has a different config map the underlying pod gets rolled.
Under the hood, Benthos csv input uses the standard encoding/csv packages's csv.Reader struct.
The current implementation of csv input doesn't allow setting the LazyQuotes field.
We have a use case where we need to set the LazyQuotes field in order to make things work correctly.
Is your feature request related to a problem? Please describe.
Currently in feature_store.yaml, we can only specify a region for DynamoDB provider. As a result, it requires an actual DynamoDB to be available when we want to do local development/testing or integration testing in a sandbox environment.
Describe the solution you'd like
A way to solve this is to let user pass an endpoint
-
Updated
Mar 7, 2022 - Python
When we show data for a metric, we currently don't include the current day's worth of data. For users just getting set up, they may only have events from today, and want to test out if the query is working, and by excluding events from 'today', they can't see results.
TODO:
- In
packages/back-end/src/services/experiments.tson line329, instead of using the current date as the value
As seen above, there is a lot of extra space around some screenshots in the docs.
We can still use the cool tool that adds the window-style border and controls (although you can also choose to remove it), but the extra space is hard on the eyes and makes the page read less fluently.
-
Updated
Feb 2, 2022
-
Updated
Aug 2, 2021 - JavaScript
When a user declares a function with the argument products (instead of product), the error looks like this:
Error rendering task "get" initialized with function "get". Got unexpected arguments: ['product']. Missing arguments: ['products']. Pass ['products'] in "params"
This is confusing because the user may end up passing products under the params section. But under this sc
-
Updated
Mar 7, 2022 - Jupyter Notebook
-
Updated
Dec 31, 2021
(1) Add docstrings to methods
(2) Covert .format() methods to f strings for readability
(3) Make sure we are using Python 3.8 throughout
(4) zip extract_all() in ingest_flights.py can be simplified with a Path parameter
-
Updated
Mar 9, 2020 - Python
if they are not class methods then the method would be invoked for every test and a session would be created for each of those tests.
`class PySparkTest(unittest.TestCase):
@classmethod
def suppress_py4j_logging(cls):
logger = logging.getLogger('py4j')
logger.setLevel(logging.WARN)
@classmethod
def create_testing_pyspark_session(cls):
return Sp
Background
This thread is borne out of the discussion from #968 , in an effort to make documentation more beginner-friendly & more understandable.
One of the subtasks mentioned in that thread was to go through the function docstrings and include a minimal working example to each of the public functions in pyjanitor.
Criteria reiterated here for the benefit of discussion:
It sh
-
Updated
Mar 4, 2022 - Dockerfile
-
Updated
Mar 7, 2022 - Python
Follow the implementation example of ingestion/tests/integration/ometa/test_ometa_database_service_api.py to implement the testing of the Python client for PipelineService.
-
Updated
Jun 2, 2021
-
Updated
Mar 5, 2020 - Python
-
Updated
Feb 2, 2022
-
Updated
Mar 7, 2022 - Python
-
Updated
Nov 6, 2021 - Ruby
Improve this page
Add a description, image, and links to the data-engineering topic page so that developers can more easily learn about it.
Add this topic to your repo
To associate your repository with the data-engineering topic, visit your repo's landing page and select "manage topics."

The Mixed Time-Series chart type allows for configuring the title of the primary and the secondary y-axis.
However, while only the title of the primary axis is shown next to the axis, the title of the secondary one is placed at the upper end of the axis where it gets hidden by bar values and zoom controls.
How to reproduce the bug