Google App Engine 批量加载器转换的参考页面在哪里?

2024-01-09

从空数据存储中,我能够自动生成bulkloader.yaml文件。它只包含python_preamble,但是transformers部分是空的。

python_preamble:
- import: google.appengine.ext.bulkload.transform
- import: google.appengine.ext.bulkload.bulkloader_wizard
- import: my_own_transformers
- import: data_models  # This is where the SomeData class is defined.
# some more imports here

然后,根据文档中的示例,我需要为 CSV 中的每一列定义一个属性映射:

transformers:
- kind: SomeData
  connector: csv
  property_map:
    - property: date
      import_transform: transform.some_undocumented_function

两个问题:

我的理解是该函数定义为import_transform会将普通的 CSV 字符串转换为财产类别 http://code.google.com/appengine/docs/python/datastore/typesandpropertyclasses.html值得数据存储。我想了解转换是如何工作的,所以我想我有两种选择。

  1. 图书馆参考在哪里google.appengine.ext.bulkload.transform?我想知道如何使用transform.some_undocumented_function,以及所有其他transform.some_other_undocumented_transformers

  2. 你可以从我的python_preamble that I - import: my_own_transformers。在该模块中,我定义了一个函数transform_date它采用 ISO 日期字符串,例如2001-01-01并将其转换为适合的类型db.DateProperty()。如果我的概念是正确的,我可以使用:



property_map:
  - property: date
  import_transform: my_own_transforms.transform_date
  

1)
您可以检查或向交互式控制台提供如下内容:

from google.appengine.ext.bulkload import transform 
help(transform)

你会得到:

Help on module google.appengine.ext.bulkload.transform in google.appengine.ext.bulkload:

NAME
    google.appengine.ext.bulkload.transform - Bulkloader Transform Helper functions.

FILE
    /Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/ext/bulkload/transform.py

DESCRIPTION
    A collection of helper functions for bulkloading data, typically referenced
    from a bulkloader.yaml file.

FUNCTIONS
    blob_to_file(filename_hint_propertyname=None, directory_hint='')
        Write the blob contents to a file, and replace them with the filename.

        Args:
          filename_hint_propertyname: If present, the filename will begin with
            the contents of this value in the entity being exported.
          directory_hint: If present, the files will be stored in this directory.

        Returns:
          A function which writes the input blob to a file.

    blobproperty_from_base64 = wrapper(value)

    bytestring_from_base64 = wrapper(value)

    child_node_from_list(child_node_name)
        Return a value suitable for generating an XML child node on export.

        The return value is a list of tuples which the simplexml connector will
        use to build a child node.

        See also list_from_child_node

        Args:
          child_node_name: The name to use for each child node.

        Returns:
          Transform function which works as described in the args.

    create_deep_key(*path_info)
        A method to make multi-level Key objects.

        Generates multi-level key from multiple fields in the input dictionary.

        This is typically used for Keys for entities which have variable parent keys,
        e.g. ones with owned relationships. It can used for both __key__ and
        references.

        Use create_foreign_key as a simpler way to create single level keys.

        Args:
          path_info: List of tuples, describing (kind, property, is_id=False).
            kind: The kind name.
            property: The external property in the current import dictionary, or
              transform.CURRENT_PROPERTY for the value passed to the transform.
            is_id: Converts value to int and treats as numeric ID if True, otherwise
              the value is a string name. Default is False.
            Example:
              create_deep_key(('rootkind', 'rootcolumn'),
                              ('childkind', 'childcolumn', True),
                              ('leafkind', transform.CURRENT_PROPERTY))

        Returns:
          Transform method which parses the info from the current neutral dictionary
          into a Key with parents as described by path_info.

    create_foreign_key(kind, key_is_id=False)
        A method to make one-level Key objects.

        These are typically used in ReferenceProperty in Python, where the reference
        value is a key with kind (or model) name name.

        This helper method does not support keys with parents. Use create_deep_key
        instead to create keys with parents.

        Args:
          kind: The kind name of the reference as a string.
          key_is_id: If true, convert the key into an integer to be used as an id.
            If false, leave the key in the input format (typically a string).

        Returns:
          Single argument method which parses a value into a Key of kind entity_kind.

    empty_if_none(fn)
        A wrapper for a value to return '' if it's None. Useful on export.

        Can be used in config files (e.g. "transform.empty_if_none(unicode)" or
        as a decorator.

        Args:
          fn: Single argument transform function.

        Returns:
          Wrapped function.

    export_date_time(format)
        A wrapper around strftime. Also returns '' if the input is None.

        Args:
          format: Format string for strftime.

        Returns:
          Single argument method which convers a datetime into a string using format.

    import_date_time(format, _strptime=None)
        A wrapper around strptime. Also returns None if the input is empty.

        Args:
          format: Format string for strptime.

        Returns:
          Single argument method which parses a string into a datetime using format.

    join_list(delimeter)
        Join a list into a string using the delimeter.

        This is just a wrapper for string.join.

        Args:
          delimeter: The delimiter to use when joining the string.

        Returns:
          Method which joins the list into a string with the delimeter.

    key_id_or_name_as_string = transform_function(key)

    key_id_or_name_as_string_n(index)
        Pull out the nth (0-based) key id or name from a key which has parents.

        If a key is present, return its id or name as a string.

        Note that this loses the distinction between integer IDs and strings
        which happen to look like integers. Use key_type to distinguish them.

        This is a useful complement to create_deep_key.

        Args:
          index: The depth of the id or name to extract. Zero is the root key.
              Negative one is the leaf key.

        Returns:
          Function extracting the name or ID of the key at depth index, as a unicode
          string. Returns '' if key is empty (unsaved), otherwise raises IndexError
          if the key is not as deep as described.

    key_kind = wrapper(value)

    key_kind_n(index)
        Pull out the nth (0-based) key kind from a key which has parents.

        This is a useful complement to create_deep_key.

        Args:
          index: The depth of the id or name to extract. Zero is the root key.
            Negative one is the leaf key.

        Returns:
          Function returning the kind of the key at depth index, or raising
          IndexError if the key is not as deep as described.

    key_type = transform_function(key)

    key_type_n(index)
        Pull out the nth (0-based) key type from a key which has parents.

        This is most useful when paired with key_id_or_name_as_string_n.
        This is a useful complement to create_deep_key.

        Args:
          index: The depth of the id or name to extract. Zero is the root key.
              Negative one is the leaf key.

        Returns:
          Method returning the type ('ID' or 'name') of the key at depth index.
          Returns '' if key is empty (unsaved), otherwise raises IndexError
          if the key is not as deep as described.

    list_from_child_node(xpath, suppress_blank=False)
        Return a list property from child nodes of the current xml node.

        This applies only the simplexml helper, as it assumes __node__, the current
        ElementTree node corresponding to the import record.

        Sample usage for structure:
         <Visit>
          <VisitActivities>
           <Activity>A1</Activity>
           <Activity>A2</Activity>
          </VisitActivities>
         </Visit>

        property: activities
        external_name: VisitActivities # Ignored on import, used on export.
        import_transform: list_from_xml_node('VisitActivities/Activity')
        export_transform: child_node_from_list('Activity')

        Args:
          xpath: XPath to run on the current node.
          suppress_blank: if True, ndoes with no text will be skipped.

        Returns:
          Transform function which works as described in the args.

    list_from_multiproperty(*external_names)
        Create a list from multiple properties.

        Args:
          external_names: List of the properties to use.

        Returns:
          Transform function which returns a list of the properties in external_names.

    none_if_empty(fn)
        A decorator which returns None if its input is empty else fn(x).

        Useful on import.  Can be used in config files
        (e.g. "transform.none_if_empty(int)" or as a decorator.

        Args:
          fn: Single argument transform function.

        Returns:
          Wrapped function.

    property_from_list(index)
        Return the Nth item from a list, or '' if the list is shorter.

        Args:
          index: Item in the list to return.

        Returns:
          Function returning the item from a list, or '' if the list is too short.

    regexp_bool(regexp, flags=0)
        Return a boolean if the expression matches with re.match.

        Note that re.match anchors at the start but not end of the string.

        Args:
          regexp: String, regular expression.
          flags: Optional flags to pass to re.match.

        Returns:
          Method which returns a Boolean if the expression matches.

    regexp_extract(pattern, method=<function match at 0x336270>, group=1)
        Return first group in the value matching the pattern using re.match.

        Args:
          pattern: A regular expression to match on with at least one group.
          method: The method to use for matching; normally re.match or re.search.
          group: The group to use for extracting a value.

        Returns:
          A single argument method which returns the group_arg group matched,
          or None if no match was found or the input was empty.

    regexp_to_list(pattern)
        Return function that returns a list of objects that match the regex.

        Useful on import.  Uses the provided regex to split a string value into a list
        of strings.  Wrapped by none_if_input_or_result_empty, so returns none if
        there are no matches for the regex and none if the input is empty.

        Args:
          pattern: A regular expression pattern to match against the input string.

        Returns:
          None if the input was none or no matches were found, otherwise a list of
          strings matching the input expression.

    split_string(delimeter)
        Split a string using the delimeter into a list.

        This is just a wrapper for string.split.

        Args:
          delimeter: The delimiter to split the string on.

        Returns:
          Method which splits the string into a list along the delimeter.

DATA
    CURRENT_PROPERTY = None
    KEY_TYPE_ID = 'ID'
    KEY_TYPE_NAME = 'name'
    __loader__ = <google.appengine.tools.dev_appserver.HardenedModulesHook...

2)
确切地说,您可以使用定义的转换函数,或者在这种特定情况下,您可以直接使用transform.import_date_time.

import_date_time(format, _strptime=None)
   A wrapper around strptime. Also returns None if the input is empty.

   Args:
     format: Format string for strptime.

   Returns:
     Single argument method which parses a string into a datetime using format.
本文内容由网友自发贡献,版权归原作者所有,本站不承担相应法律责任。如您发现有涉嫌抄袭侵权的内容,请联系:hwhale#tublm.com(使用前将#替换为@)

Google App Engine 批量加载器转换的参考页面在哪里? 的相关文章

  • 将非 GAC 引用添加到项目中

    每次我在 Visual Studio 2008 中添加对 GAC 中的 Web 项目的引用时 它都会将该引用添加为 GAC 引用 并且不会将该文件复制到我的 bin 目录中 但出于部署目的 我想将引用添加为非 GAC 引用 以便将 dll
  • 引用的地址总是等于原始地址吗?

    这看起来是一个非常基本的话题 但对我来说非常重要 以下示例显示引用变量的地址等于原始变量的地址 我知道这就是我们从 C C 的概念中可以期待的 然而 是不是始终保证这些地址在任何情况下都是相等的吗 include
  • 视频文件转换/转码 Google App Engine

    我想启动一个云计算项目 其简单任务是 接收上传的视频文件 对它们进行一些转码 转换 允许用户下载 流式传输生成的文件 我刚在想ffmpeg作为集成在的外部命令行工具Java Google App engine Application 由于很
  • 会话超时(session.setMaxInactiveInterval)在 Google Appengine 中不起作用

    我尝试使用以下命令为特定用户设置会话超时session setMaxInactiveInterval 30 60 30分钟 但默认情况下它被分配为 86400 秒 24 小时 也在 Web xml 中尝试过
  • 将 ical 附件的邮件消息的内容类型设置为“text/calendar; method=REQUEST”

    我正在尝试使用 App Engine 邮件 API 从 App Engine 发送 iCalendar 格式的 ics 文件 这在 GMail 中非常有效 但是 Outlook 无法识别该文件 我认为问题在于内容类型设置为 文本 日历 而不
  • Ruby on Rails 基本概念总结

    作为 Rails 的新手 我很难找到提供 Ruby on Rails 概要的网站或参考资料 我对 MVC ActiveRecord 以及诸如此类的东西有基本的了解 但我很难理解其中的一些关系和基础知识 例如 我需要注意哪些命名约定 控制器操
  • 为什么在闭包参数中使用“&&”?

    我有两个问题这个例子 https doc rust lang org std iter trait Iterator html method find let a 1 2 3 assert eq a iter find x x 2 Some
  • 找不到参考组件 Microsoft.CSharp

    我在 Visual Studio 2010 启动的 C 项目上发现了这个问题 当我转到 2008 年使用的另一台 PC 时 我打开 project csproj 需要 get 或 set 访问器 和警告 找不到引用的组件 Microsoft
  • css 变换,chrome 中的锯齿状边缘

    我一直在使用 CSS3 转换来旋转网站中带有边框的图像和文本框 问题是 Chrome 中的边框看起来呈锯齿状 就像没有抗锯齿的 低分辨率 游戏一样 在 IE Opera 和 FF 中看起来好多了 因为使用了 AA 仍然清晰可见 但也没有那么
  • DatastoreFailureException:内部错误

    刚刚开始收到此错误 没有进行任何代码更改 数据存储上的 GAE J Datanucleus JDO JPA 版本 v1 我在欧盟服务器上的应用程序无法运行 美国目前不受影响 怀疑谷歌内部有什么问题 有人能同意吗 是的 大约 40 分钟前我开
  • 在 Google App Engine 中使用 sqlite3?

    我正在尝试将我的 Python Django 项目部署到 Google App Engine 现在它在我的本地计算机上运行良好 但是当我尝试将其作为 Google App Engine 中的项目运行时 出现以下错误 ImproperlyCo
  • 在objectify中查询字段不为空的所有记录的有效方法

    我想有效地查询索引字段以检索索引字段不为空 存在于索引中 的所有记录 需要查询的字段包含一个Ref
  • Golang Appengine 项目无法构建

    我有一个使用 golang 的应用程序引擎项目 我已经大约一年没有碰过了 我现在无法让它在之前构建的机器上构建 我收到以下错误 go app builder 解析输入失败 解析器 src golang org x net internal
  • 为什么转发引用与右值引用具有相同的语法?

    我刚刚对这些 相当 新的功能做了一些研究 我想知道为什么 C 委员会决定为它们引入相同的语法 看来开发人员不必要浪费一些时间来理解它是如何工作的 而一种解决方案可以让我们思考进一步的问题 就我而言 它是从问题开始的 可以简化为 includ
  • App Engine 部署失败,并显示“客户端错误 (400) 由于未指定的原因,请求无效。”

    我的 python27 应用程序部署由于某些原因失败 2012 03 09 16 46 25 Running command C Python27 pythonw exe u C Program Files x86 Google googl
  • GAE SDK 1.9.5 和 InvalidCertificateException

    尝试在 osX Maverics 10 9 5 上使用 Python2 7 8 从 GAE SDK 1 95 导入测试床 我收到 InvalidCertificateException 错误 from google appengine ex
  • gradlew:appengineEnhance 失败

    我正在使用 Java 创建移动后端Google App Engine with Android Studio 为了启动公开我的 API 的本地服务器 我使用gradlew module name appengineRun 然而 当我去htt
  • 配置 Restlet 以在 Google App Engine 上返回 JSP?

    我开发了一个 Restlet 应用程序 我想通过 Restlet 返回一个 URL 请求的 JSP 文件 在不使用重定向的情况下如何实现此目的 IE 假设我在 mydomain com 上有文件 contact jsp 并且我希望人们能够访
  • 如何确定透视变换后的点在新图像平面中的位置?

    我使用 OpenCV Python Numpy 图像中有三个点 我知道这些点的确切位置 P1 P2 N1 我要将图像转换为另一个视图 例如 我将透视图转换为侧视图 如果这样做 我将无法获得图像平面中这三个点的确切位置 我应该以一种可以获得这
  • API 调用 datastore_v3.Put() 的请求太大

    我正在使用 google cloud sql 和 appengine 我正进入 状态com google apphosting api ApiProxy RequestTooLargeException The request to API

随机推荐