data-engineering
Here are 679 public repositories matching this topic...
-
Updated
Feb 5, 2021
Current behavior
All ShellTask logs are logged at DEBUG level, preventing showing the command output in the UI logs unless the agent's logging threshold is set to DEBUG. But this is undesired for production workloads, as this applies to all log messages
streaming logging for the ShellTask was introduced in #3584
Proposed behavior
change existing stream_logs parameter to `U
-
Updated
Jan 15, 2021
Describe the bug
When trying to run scaffolding (profiling) command, it fails because of commas in columns.
To Reproduce
Steps to reproduce the behavior:
- Run
great_expectations suite scaffold scaffold-nameon datasource where commas are in column - Bug
pandas.errors.ParserError: Error tokenizing data. C error: Expected 1 fields in line 5323 saw 2
Expected behavior
D
-
Updated
Feb 6, 2021 - Go
-
Updated
Jan 13, 2021
-
Updated
Jan 18, 2021 - JavaScript
-
Updated
Feb 4, 2021 - Python
-
Updated
Feb 6, 2021 - Jupyter Notebook
-
Updated
Feb 2, 2021 - Jupyter Notebook
-
Updated
Mar 9, 2020 - Python
Enable delete repository action from the UI
Problem description
When I use the function of concatenating multiple columns, I find that it does not handle null values as expected.
This is the current output
df.concatenate_columns(["cat_1","cat_2","cat_3"],"cat",sep=",")| cat_1 | cat_2 |
|---|
if they are not class methods then the method would be invoked for every test and a session would be created for each of those tests.
`class PySparkTest(unittest.TestCase):
@classmethod
def suppress_py4j_logging(cls):
logger = logging.getLogger('py4j')
logger.setLevel(logging.WARN)
@classmethod
def create_testing_pyspark_session(cls):
return Sp
-
Updated
Feb 6, 2021 - R
-
Updated
Jan 17, 2021
-
Updated
Mar 5, 2020 - Python
-
Updated
Jan 15, 2021 - Ruby
-
Updated
Jan 20, 2021 - CSS
-
Updated
Jan 15, 2021
-
Updated
Feb 5, 2021 - TypeScript
-
Updated
Nov 29, 2018 - Java
-
Updated
Feb 5, 2021 - Java
-
Updated
Apr 20, 2020 - Python
-
Updated
Feb 3, 2021
-
Updated
Jan 4, 2021 - Python
-
Updated
Feb 3, 2021
-
Updated
Dec 4, 2020 - Python
-
Updated
Jan 26, 2021 - Python
Improve this page
Add a description, image, and links to the data-engineering topic page so that developers can more easily learn about it.
Add this topic to your repo
To associate your repository with the data-engineering topic, visit your repo's landing page and select "manage topics."
Is your feature request related to a problem? Please describe.

Main Datetime Column exists in the Legacy Datasource Editor but not in the new editor
Describe the solution you'd like
Add the field in the new Edit Dataset modal, maybe under the Settings tab Advanced sect