Azure Blob 存储 blob 到索引

2024-03-25

是否可以将文档上传到 Blob 存储并执行以下操作:

  1. 获取文档内容并添加到索引。
  2. 从第 1 点的内容中抓取关键短语并添加到索引中。

我希望关键短语可以被搜索。

我有代码可以将文档上传到 blobstorage,效果非常好,但获得此索引的唯一方法(据我所知)是使用 Azure 搜索服务中的“导入数据”,该服务使用预定义字段创建索引 -如下:

当只需要这些字段并且索引每 5 分钟自动更新一次时,这非常有效。但是当我想要自定义索引时就成了问题

然而,我唯一想要的字段如下:

  • fileId
  • fileText(这是文档的内容)
  • blobURL(允许下载文档)
  • keyPhrases(将从 fileText 中提取 - 我也有执行此操作的代码)

我遇到的唯一问题是我需要能够检索文档内容(fileText)才能获取关键短语,但据我了解,只有当文档内容已经在索引中时我才能执行此操作访问该内容?

我对 Azure 的了解非常有限,并且很难找到与我想做的事情类似的事情。

我用来将文档上传到我的 blob 存储的代码如下:

public CloudBlockBlob UploadBlob(HttpPostedFileBase file)
    {
        string searchServiceName = ConfigurationManager.AppSettings["SearchServiceName"];
        string blobStorageKey = ConfigurationManager.AppSettings["BlobStorageKey"];
        string blobStorageName = ConfigurationManager.AppSettings["BlobStorageName"];
        string blobStorageURL = ConfigurationManager.AppSettings["BlobStorageURL"];
        string UserID = User.Identity.GetUserId();
        string UploadDateTime = DateTime.Now.ToString("yyyyMMddhhmmss").ToString();

        try
        {
            var path = Path.Combine(Server.MapPath("~/App_Data/Uploads"), UserID + "_" + UploadDateTime + "_" + file.FileName);

            file.SaveAs(path);

            var credentials = new StorageCredentials(searchServiceName, blobStorageKey);

            var client = new CloudBlobClient(new Uri(blobStorageURL), credentials);

            // Retrieve a reference to a container. (You need to create one using the mangement portal, or call container.CreateIfNotExists())
            var container = client.GetContainerReference(blobStorageName);

            // Retrieve reference to a blob named "myfile.gif".
            var blockBlob = container.GetBlockBlobReference(UserID + "_" + UploadDateTime + "_" + file.FileName);

            // Create or overwrite the "myblob" blob with contents from a local file.
            using (var fileStream = System.IO.File.OpenRead(path))
            {
                blockBlob.UploadFromStream(fileStream);
            }

            System.IO.File.Delete(path);

            return blockBlob;
        }
        catch (Exception e)
        {
            var r = e.Message;
            return null;
        }
    }

我希望我没有提供太多信息,但我不知道如何解释我正在寻找的内容。如果我不明白,请告诉我,以便我解决我的问题。

我不是在寻找讲义代码,只是在寻找正确方向的推动力。

我将不胜感激任何帮助。

Thanks!


我们可以使用Azure搜索通过Azure搜索来索引文档REST API https://learn.microsoft.com/zh-cn/rest/api/searchservice/Indexer-operations and .NET SDK https://learn.microsoft.com/en-us/azure/search/search-howto-dotnet-sdk。 根据你的描述,我用.NET SDK创建了一个demo并测试成功。以下是我的详细步骤:

  1. 从 Azure 门户创建 Azure 搜索
  1. 从 Azure 门户获取搜索密钥
  1. 创建自定义索引字段模型

    [SerializePropertyNamesAsCamelCase] public class TomTestModel { [Key] [IsFilterable] public string fileId { get; set; } [IsSearchable] public string fileText { get; set; } public string blobURL { get; set; } [IsSearchable] public string keyPhrases { get; set; } }

4.创建数据源

       string searchServiceName = ConfigurationManager.AppSettings["SearchServiceName"];
       string adminApiKey = ConfigurationManager.AppSettings["SearchServiceAdminApiKey"];
       SearchServiceClient serviceClient = new SearchServiceClient(searchServiceName, new SearchCredentials(adminApiKey));

       var dataSource = DataSource.AzureBlobStorage("storage name", "connectstrong", "container name");
        //create data source
        if (serviceClient.DataSources.Exists(dataSource.Name))
        {
            serviceClient.DataSources.Delete(dataSource.Name);
        }
        serviceClient.DataSources.Create(dataSource);
  1. 创建自定义索引

var definition = new Index() { Name = "tomcustomindex", Fields = FieldBuilder.BuildForType<TomTestModel>() }; //create Index if (serviceClient.Indexes.Exists(definition.Name)) { serviceClient.Indexes.Delete(definition.Name); } var index = serviceClient.Indexes.Create(definition);

  1. 上传文档到索引,更多关于使用SDK操作存储的信息请参考document https://learn.microsoft.com/en-us/azure/storage/storage-dotnet-how-to-use-blobs#download-blobs

            CloudStorageAccount storageAccount = CloudStorageAccount.Parse("connection string");
            var blobClient = storageAccount.CreateCloudBlobClient();
            var container =blobClient.GetContainerReference("container name");
            var blobList = container.ListBlobs();
    
            var tomIndexsList = blobList.Select(blob => new TomTestModel
            {
                fileId = Guid.NewGuid().ToString(), blobURL = blob.Uri.ToString(), fileText = "Blob Content", keyPhrases = "key phrases",
            }).ToList();
            var batch = IndexBatch.Upload(tomIndexsList);
            ISearchIndexClient indexClient = serviceClient.Indexes.GetClient("index");
            indexClient.Documents.Index(batch);
    
  2. 检查搜索探索的搜索结果。

页面.config 文件:

<?xml version="1.0" encoding="utf-8"?>
<packages>
  <package id="Microsoft.Azure.KeyVault.Core" version="1.0.0" targetFramework="net452" />
  <package id="Microsoft.Azure.Search" version="3.0.0-rc" targetFramework="net452" />
  <package id="Microsoft.Data.Edm" version="5.6.4" targetFramework="net452" />
  <package id="Microsoft.Data.OData" version="5.6.4" targetFramework="net452" />
  <package id="Microsoft.Data.Services.Client" version="5.6.4" targetFramework="net452" />
  <package id="Microsoft.Rest.ClientRuntime" version="2.3.4" targetFramework="net452" />
  <package id="Microsoft.Rest.ClientRuntime.Azure" version="3.3.4" targetFramework="net452" />
  <package id="Microsoft.Spatial" version="6.15.0" targetFramework="net452" />
  <package id="Newtonsoft.Json" version="7.0.1" targetFramework="net452" />
  <package id="System.Spatial" version="5.6.4" targetFramework="net452" />
  <package id="WindowsAzure.Storage" version="7.2.1" targetFramework="net452" />
</packages>

TomTest模型文件:

using System.ComponentModel.DataAnnotations;
using Microsoft.Azure.Search;
using Microsoft.Azure.Search.Models;

namespace TomAzureSearchTest
{
    [SerializePropertyNamesAsCamelCase]
    public class TomTestModel
    {
        [Key]
        [IsFilterable]
        public string fileId { get; set; }
        [IsSearchable]
        public string fileText { get; set; }
        public string blobURL { get; set; }
        [IsSearchable]
        public string keyPhrases { get; set; }
    }
}
本文内容由网友自发贡献,版权归原作者所有,本站不承担相应法律责任。如您发现有涉嫌抄袭侵权的内容,请联系:hwhale#tublm.com(使用前将#替换为@)

Azure Blob 存储 blob 到索引 的相关文章

随机推荐