In this post, I’ll show you how to build an AI-powered, searchable video archive using machine learning and Google Cloud-no experience required.

在本文中，我将向您展示如何使用机器学习和Google Cloud(无需任何经验)来构建基于AI的可搜索视频档案。

One of my favorite apps ever is definitely Google Photos. In addition to backing up my precious pics to the cloud, it also makes all of my photos and videos searchable using machine learning. So if I type “pool” in the Photos app, it returns all everything it recognizes as a pool:

我最喜欢的应用程序之一肯定是Google相册。 除了将我的珍贵照片备份到云之外，它还使我可以使用机器学习搜索所有照片和视频。 因此，如果我在“照片”应用中输入“ pool”，它将返回它识别为泳池的所有内容：

This is all well and good if you just want to use somebody else’s software, but what fun is that? Today we’ll build our own version of Google Photos, for videos.

如果您只想使用别人的软件，这一切都很好，但是那有什么乐趣？ 今天，我们将为视频构建自己的Google相册版本。

Not for nothing, there are lots of good reasons to build your own video archive. For one, it’s fun. For two, you can add features Google Photos doesn’t currently support, especially for videos. Like searching by what people say (transcripts), in case you need to find all the clips where someone says, “well now we have it on film,” or “oh sh*t.” For three, building your own app allows you to more easily integrate with your other software and control how your data is stored and handled. For example, I built my archive’s backend on Google Cloud, which let me take advantage of Google Cloud’s privacy, security, and compliance guarantees.

并非没有，有很多充分的理由来建立自己的视频档案。 首先，这很有趣。 对于其中两个，您可以添加Google相册当前不支持的功能，尤其是对于视频。 就像搜索别人说的文字一样，如果您需要找到所有有人说“现在我们在电影中”或“哦，哦，哦”的片段，就可以了。 对于三个人，构建自己的应用程序使您可以更轻松地与其他软件集成，并控制如何存储和处理数据。 例如，我在Google Cloud上构建了归档的后端，这使我可以利用Google Cloud的隐私，安全性和合规性保证 。

My searchable video archive ended up looking like this:

我的可搜索视频档案最终看起来像这样：

and it stored and indexed all of my family home videos (~126 GB). Using machine learning, specifically the Video Intelligence API, I was able to do all sorts of analysis, including automatically splitting long videos, identifying objects and scenes, transcribing audio, and extracting on-screen text.

它存储并索引了我所有的家庭视频(〜126 GB)。 使用机器学习，特别是视频智能API ，我能够进行各种分析，包括自动分割长视频，识别对象和场景，转录音频以及提取屏幕上的文本。

The app ended up being extremely good at searching for cute moments. Using computer vision, it recognized scenes and objects like “wedding,” “firework”, “performance,” “baby laughing”, “home improvement,” “roller coaster,” and even “Disney World”:

该应用程序最终非常擅长搜索可爱的时刻。 使用计算机视觉，它可以识别场景和对象，例如“婚礼”，“烟花”，“表演”，“婴儿笑”，“家庭装修”，“过山车”，甚至是“迪士尼世界”：

It could also search transcripts. This is how I found the clip of my very first steps, because in these clips, my parents say something like, “Look, Dale is taking her first steps!”:

它也可以搜索成绩单。 这就是我找到第一步的片段的方式，因为在这些片段中，我的父母说：“看，Dale正在迈出她的第一步！”：

Finally, the tool was able to search any on-screen text, like the words “Mets” and “New York” on these players’ shirts or the “Bud” poster in the background:

最终，该工具能够搜索任何屏幕上的文本，例如这些球员的球衣上的“大都会”和“纽约”或背景中的“ Bud”海报。

The video archive ended up being a pretty good Father’s Day gift, especially since I wasn’t actually able to see my dad in person this year.

该视频档案最终成为了父亲节很好的礼物，尤其是因为我今年实际上不能亲自见到父亲。

In this post, I’ll show you how you can build your own archive, just like this. But if you want to skip straight to the code, check out the Making with ML Github repo.

在这篇文章中，我将向您展示如何构建自己的存档，就像这样。 但是，如果您想直接跳到代码，请查看ML Github回购制作。

视频处理的机器学习架构 (Machine Learning Architecture for Video Processing)

The app is divided into two bits, the frontend and the backend. The backend was built using a combination of Google Cloud, Firebase, and a tool called Algolia (for search). The frontend was built with Flutter, a framework for building web and mobile apps, but could have easily been a React or Angular or iOS or Android app.

该应用程序分为两个部分，前端和后端。 后端是使用Google Cloud ， Firebase和称为