如果我理解正确的话,您希望能够获取 ADF 的 Azure Monitor 日志并查询/存储这些日志吗?
好消息是您想要查看的大部分信息已通过 Azure Monitor 收集。
提取信息的简单方法之一是使用 Azure Monitor REST API。然后,您可以将响应存储到文件或表中,或者您可以只查询 api 以获取特定管道或触发器等。
以下是授权示例和使用 Azure Monitor API 的链接:https://learn.microsoft.com/en-us/azure/azure-monitor/essentials/rest-api-walkthrough#authenticating-azure-monitor-requests https://learn.microsoft.com/en-us/azure/azure-monitor/essentials/rest-api-walkthrough#authenticating-azure-monitor-requests
这是使用 Azure Monitor REST API 获取活动运行数据的 HTTP URL 示例(动态内容语法):
@{concat('https://management.azure.com/subscriptions/', linkedService().SubscriptionID, '/resourceGroups/', linkedService().ResourceGroupName, '/providers/Microsoft.DataFactory/factories/', linkedService().DataFactoryName, '/pipelineruns/', linkedService().RunID, '/queryActivityruns?api-version=2018-06-01')}
以下是可以从 Azure Monitor 中提取的所有不同的 ADF 指标:https://learn.microsoft.com/en-us/azure/data-factory/monitor-using-azure-monitor#data-factory-metrics https://learn.microsoft.com/en-us/azure/data-factory/monitor-using-azure-monitor#data-factory-metrics
Here is an example of a REST Linked service in ADF that can be used to call the rest api:
You could then create a dataset that will pass all the values to the linked service so that you can call the API, and copy the response to a DB, like here:
This particular example is getting the status of a specific Pipeline RunID, but it can be much more broad than that.
Here is what the request body looks like so that I could filter to just failed pipelines in the last day: