如何在 Azure 数据工厂 - Databricks 中使用 continuation_token 获取 ADF Pipeline 运行详细信息的下一页?

2024-05-17

我在用

adf_client.pipeline_runs.query_by_factory(resourceGroupName, 工厂名称、过滤器参数)

的方法azure.mgmt.datafactory.DataFactoryManagementClient用于获取 ADF Pipeline Run 详细信息的包。

上述函数的响应一次返回 100 条管道运行记录。随着响应,它返回延续令牌我相信,它应该用于获取下一组/下一页记录。

我不确定要使用哪个函数来实现此目的。我尝试使用azure.mgmt.datafactory.models.PipelineRun()函数(反复试验)以查看是否满足要求。不幸的是,事实并非如此。 MS 文档也非常抽象,难以理解。

那么,Azure的Python SDK中的哪个函数可以用来获取下一页的运行记录呢?


from azure.common.credentials import ServicePrincipalCredentials
from azure.mgmt.resource import ResourceManagementClient
from azure.mgmt.datafactory import DataFactoryManagementClient
from azure.mgmt.datafactory.models import *
from datetime import datetime, timedelta
from azure.identity import ClientSecretCredential 


subscription_id = "b83c1wd3-xxxx-xxxx-xxxx-2b83a074c23f"
rg_name = "My-rg"
df_name = "ktestadf"

tenant_id = "12f978bf-xxxx-xxxx-xxxx-2d7cd011db47"
client_id = "a71ad3ca-xxxx-xxxx-xxxx-af0c2a3fdae1"
client_secret = "Nym7Q~j5YMyxxxxxx3tAk879y9vLrxAQqaI8n"

credential = ClientSecretCredential(
tenant_id=tenant_id ,
client_id=client_id ,
client_secret=client_secret)
adf_client = DataFactoryManagementClient( credential=credential,
subscription_id=subscription_id)

pipe_run = []
dsb = datetime.now()
dsa = dsb - timedelta(hours = 24)
filter_params = RunFilterParameters(last_updated_after=dsa, last_updated_before=dsb)
pipeline_runs = adf_client.pipeline_runs.query_by_factory(resource_group_name=rg_name, factory_name=df_name, filter_parameters = filter_params)
pipe_run.append(pipeline_runs.value)
while (pipeline_runs.continuation_token):
   filter_params = RunFilterParameters(last_updated_after=dsa, last_updated_before=dsb,continuation_token = pipeline_runs.continuation_token)
   pipeline_runs = adf_client.pipeline_runs.query_by_factory(
   resource_group_name=rg_name, factory_name=df_name, filter_parameters=filter_params)
   pipe_run.append(pipeline_runs.value)
本文内容由网友自发贡献,版权归原作者所有,本站不承担相应法律责任。如您发现有涉嫌抄袭侵权的内容,请联系:hwhale#tublm.com(使用前将#替换为@)

如何在 Azure 数据工厂 - Databricks 中使用 continuation_token 获取 ADF Pipeline 运行详细信息的下一页? 的相关文章

随机推荐