DataOps.live Products

Hi,We had an issue where are runner VM was full and we needed to clear down some files.  Since then however whenever I have tried to build a pipeline that creates functions where the return type is a table and the field names contain spaces e.g. :        return_type: TABLE(                           "DIVISION CODE" TEXT,                           "DIVISION" TEXT,                           "YPLAN" NUMBER) The pipeline fails like so : [ DATABASE-LEVEL ] - [ INFO] - [12:11:06.0955] - Starting Validation on generated configuration329╷330│ Error: invalid data type: CODE" TEXT with snowflake_function_sql.REPORTING_FB_DEV2_REPORTING_GETSALESBYWAREHOUSESTOREWTDANDTD,344│ on resources.tf.json line 669, in [15].resource.snowflake_function_sql.REPORTING_FB_DEV2_REPORTING_GETSALESBYWAREHOUSESTOREWTDANDTD:345│ 669: "return_type": "TABLE( \"DIVISION CODE\" TEXT, \"DIVISION\" TEXT, \"YPLAN\" NUMBER)", Replacing the space within the column name with an underscore fixes this, but that would result more

We have a parent-child pipeline (one simply calling another in a different project like so)Trigger TransformationPipeline: stage: Downstream inherit: variables: false variables: _PIPELINE_FILE_NAME: ingestion_15Min-ci.yml trigger: project: "/transformation/transformation-template" branch: $CI_COMMIT_REF_NAME and I would like to pass a variable downstream.The documentation doesnt make it exactly clear how to do this, and the gitlab says it has a keyword of trigger:forward So just after the correct way to do this, and possibly to confirm how the variable would be used in the child:ingestion_15Min-ci.yml:include: - /pipelines/includes/bootstrap.yml - /pipelines/includes/local_includes/ingestion/clear_staging.yml - /pipelines/includes/local_includes/ingestion/direct_ingestion_orchestrator.yml ## MATE pipeline content - /mate_15Min-ci.ymlmate_15Min-ci.yml:- /pipelines/includes/local_includes/modelling_and_transformation/build_15_models.yml build_15_models.yml:Build a

Confused, There is no clear guidance on how to create a free trail account?If any one know about this, please help me. 

Dear Dataops.live community, I’m new to DataOps.live and eager to learn the platform.Would it be possible to get access to a free trial account to support my training?Thank you for your consideration.Best regards,Ahmed

We want to use DataOps to load data from Azure Storage on a regular basis (every 30 minutes).  The data is stored in blob storage as datetimestamp/entity/csv file e.g. : /2024-07-08T09.14.53Z/exchangerate/2024.csv/2024-07-08T09.30.04Z/exchangerate/2024.csv/2024-07-30T13.29.53Z/exchangerate/2024.csv To use the pattern method takes too long, so I was wondering if it possible to use a before script to get a list of valid files (something along using the snowflake list @stage command) and then create that as a variable and then place that into the path e.g. path : <variable> but I cant see any documentation / help online that shows this?

I am trying to get DO to create a SP and in it, it needs to be execute as caller,  but when it is getting built it is coming out as OwnerAnd it does need to be as Caller.

Hi all.I’ve been trying to restrict the runners secret selection by adding the environment config named SECRETS_SELECTION to:/srv/dataops-runner-abcd-latest/config/config.toml The configuration setting is:environment = ["SECRETS_SELECTION=dataops/abcd"] In the DataOps.live project the variables.yaml file is set to ask for secrets:SECRETS_MANAGER: AWS_PARAMETER_STORESECRETS_SELECTION: /dataops/xxxx/SNOWFLAKE/SOLE/SECRETS_STRIP_PREFIX: /dataops/xxxx/ But when I trigger a pipeline that uses the restricted runner the pipeline can still access the secrets XXXX.What am I doing wrong?Many thanks.Dave. 

Hello,How does the new feature "Community Help" search in Spendview, sets the relevance of search results?And is it hitting result from Spendview documentation portal?Thanks,Ilja

Are there any features in Spendview to alert administrators of suspicious activities?Thanks,Ilja

Are you having problem with logging in ? here are couple of things to consider. For those who registers your username is your email address.  Please consider that username is also case sensitive.  Dear Community, please post your questions and we’ll have crowdsourced KB !  

What to do if you keep getting an error connecting snowflake account in Spendview ?