Skip to main content

Solved

7 Topics
Ash B
Ash BNew Participant
asked in Have a question?
Stage Ingestion - Path variable

We want to use DataOps to load data from Azure Storage on a regular basis (every 30 minutes).  The data is stored in blob storage as datetimestamp/entity/csv file e.g. : /2024-07-08T09.14.53Z/exchangerate/2024.csv/2024-07-08T09.30.04Z/exchangerate/2024.csv/2024-07-30T13.29.53Z/exchangerate/2024.csv To use the pattern method takes too long, so I was wondering if it possible to use a before script to get a list of valid files (something along using the snowflake list @stage command) and then create that as a variable and then place that into the path e.g. path : <variable> but I cant see any documentation / help online that shows this?

771
sean-dataops
Ash B
Ash BNew Participant
asked in Have a question?
DO - Stored Procedure - Execute as not working as exepcted

I am trying to get DO to create a SP and in it, it needs to be execute as caller,  but when it is getting built it is coming out as OwnerAnd it does need to be as Caller.

362
Ash B
A
Anonymous
asked in #TrueDataOps Discussions
How to pass Snowflake credentials to DevReady environment?

Hey @DataOps.live ,I am curious to know more about how I can “connect” or pass my snowflake credentials to the DevReady environment and in particular a streamlit app that I am using. Would you kindly share more? Thanks

601
luke
Employee
David Richardson
David RichardsonNew Participant
asked in Have a question?
restricting a runners secret selection

Hi all.I’ve been trying to restrict the runners secret selection by adding the environment config named SECRETS_SELECTION to:/srv/dataops-runner-abcd-latest/config/config.toml The configuration setting is:environment = ["SECRETS_SELECTION=dataops/abcd"] In the DataOps.live project the variables.yaml file is set to ask for secrets:SECRETS_MANAGER: AWS_PARAMETER_STORESECRETS_SELECTION: /dataops/xxxx/SNOWFLAKE/SOLE/SECRETS_STRIP_PREFIX: /dataops/xxxx/ But when I trigger a pipeline that uses the restricted runner the pipeline can still access the secrets XXXX.What am I doing wrong?Many thanks.Dave. 

843
Traycho Milev
ilja.tarans
ilja.taransParticipating Frequently
asked in Have a question?
"Community Help" search in Spendview relevance results

Hello,How does the new feature "Community Help" search in Spendview, sets the relevance of search results?And is it hitting result from Spendview documentation portal?Thanks,Ilja

681
jevgenijs.jelistratovs
ilja.tarans
ilja.taransParticipating Frequently
asked in Have a question?
Spendview logging and monitoring features

Are there any features in Spendview to alert administrators of suspicious activities?Thanks,Ilja

823
jevgenijs.jelistratovs
jevgenijs.jelistratovs
jevgenijs.jelistratovsInspiring
asked in Have a question?
Spendview Snowflake connection checklist

What to do if you keep getting an error connecting snowflake account in Spendview ? 

1202
jevgenijs.jelistratovs

Badge winners

Show all badges
Terms & ConditionsCookie settings