Skip to content

chore(migration): Migrate code from googleapis/python-db-dtypes-pandas into packages/db-dtypes#16012

Draft
parthea wants to merge 216 commits intomainfrom
migration.Python-db-dtypes-pandas.migration.2026-03-02_20-30-38.migrate
Draft

chore(migration): Migrate code from googleapis/python-db-dtypes-pandas into packages/db-dtypes#16012
parthea wants to merge 216 commits intomainfrom
migration.Python-db-dtypes-pandas.migration.2026-03-02_20-30-38.migrate

Conversation

@parthea
Copy link
Contributor

@parthea parthea commented Mar 2, 2026

See #11017.

This PR should be merged with a merge-commit, not a squash-commit, in order to preserve the git history.

tswast and others added 30 commits September 20, 2021 16:54
* test: disable system test session

* 🦉 Updates from OwlBot

See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md

* 🦉 Updates from OwlBot

See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md

* add pandas to intersphinx

* 🦉 Updates from OwlBot

See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md

* actually use dependencies

* add constraints

* fix conflict with numpy

* bump minimum packaging

Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>
* test: Don't use the equal_nan option of array_equal. It requires new versions of numpy

* 🦉 Updates from OwlBot

See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md

* blacken

* fix: support pandas 0.24

* blacken

* remove 'stop on first error'

* 🦉 Updates from OwlBot

See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md

* 🦉 Updates from OwlBot

See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md

Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>
Co-authored-by: Tim Swast <swast@google.com>
Thank you for opening a Pull Request! Before submitting your PR, there are a few things you can do to make sure it goes smoothly:
- [x] Make sure to open an issue as a [bug/issue](https://github.com/googleapis/python-db-dtypes-pandas/issues/new/choose) before writing your code!  That way we can discuss the change, evaluate designs, and agree on the general idea
- [x] Ensure the tests and linter pass
- [x] Code coverage does not decrease (if any source code was changed)
- [x] Appropriate docs were updated (if necessary)

Fixes #2 🦕

This builds-on/includes #8, which builds-on/includes #7.
* fix: support converting empty `time` Series to pyarrow Array

* use object dtype for time numpy array

* backport to_numpy

* remove redundant test
…22)

* chore: add default_version and codeowner_team to .repo-metadata.json

* update codeowner_team
* fix: avoid rounding problems with microseconds

* add more tests

* adjust regex to parse fraction as integer
…le (#25)

Source-Link: googleapis/synthtool@a7ed11e
Post-Processor: gcr.io/cloud-devrel-public-resources/owlbot-python:latest@sha256:6e7328583be8edd3ba8f35311c76a1ecbc823010279ccb6ab46b7a76e25eafcc

Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>
* docs: add how-to guide and include API reference

* fix indentation

* fix types in sample

* fix types in sample
Co-authored-by: release-please[bot] <55107282+release-please[bot]@users.noreply.github.com>
#35)

* chore(python): push cloud library docs to staging bucket for Cloud RAD

Source-Link: googleapis/synthtool@7fd61f8
Post-Processor: gcr.io/cloud-devrel-public-resources/owlbot-python:latest@sha256:4ee57a76a176ede9087c14330c625a71553cf9c72828b2c0ca12f5338171ba60

* update replacement in owlbot.py

* 🦉 Updates from OwlBot

See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md

* 🦉 Updates from OwlBot

See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md

Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>
Co-authored-by: Anthonios Partheniou <partheniou@google.com>
Source-Link: googleapis/synthtool@694118b
Post-Processor: gcr.io/cloud-devrel-public-resources/owlbot-python:latest@sha256:ec49167c606648a063d1222220b48119c912562849a0528f35bfb592a9f72737

Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>
[![WhiteSource Renovate](https://app.renovatebot.com/images/banner.svg)](https://renovatebot.com)

This PR contains the following updates:

| Package | Change | Age | Adoption | Passing | Confidence |
|---|---|---|---|---|---|
| [pyarrow](https://arrow.apache.org/) | ` >= 3.0.0, < 6.0dev` -> `>=3.0.0, <6.1` | [![age](https://badges.renovateapi.com/packages/pypi/pyarrow/6.0.0/age-slim)](https://docs.renovatebot.com/merge-confidence/) | [![adoption](https://badges.renovateapi.com/packages/pypi/pyarrow/6.0.0/adoption-slim)](https://docs.renovatebot.com/merge-confidence/) | [![passing](https://badges.renovateapi.com/packages/pypi/pyarrow/6.0.0/compatibility-slim/5.0.0)](https://docs.renovatebot.com/merge-confidence/) | [![confidence](https://badges.renovateapi.com/packages/pypi/pyarrow/6.0.0/confidence-slim/5.0.0)](https://docs.renovatebot.com/merge-confidence/) |

---

### Configuration

📅 **Schedule**: At any time (no schedule defined).

🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied.

♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

🔕 **Ignore**: Close this PR and you won't be reminded about this update again.

---

 - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, click this checkbox.

---

This PR has been generated by [WhiteSource Renovate](https://renovate.whitesourcesoftware.com). View repository job log [here](https://app.renovatebot.com/dashboard#github/googleapis/python-db-dtypes-pandas).
* feat: support Python 3.10

* 🦉 Updates from OwlBot

See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md

* 🦉 Updates from OwlBot

See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md

Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>
…#39)

* feat: support conversion from pyarrow RecordBatch to pandas DataFrame

* hack together working implementation

TODO: add tests for constructing pandas Series with pyarrow scalars

* fix unit test coverage, optimize arrow to numpy conversion

* apply same optimizations to to_arrow conversion

* remove redundant to_numpy now that to_arrow doesn't use it

* be explicit about chunked array vs array

* add docstrings to arrow conversion functions

* add test case for round-trip to/from pyarrow nanosecond-precision time scalars

* add time32("ms") test case without nulls for completeness
Source-Link: googleapis/synthtool@c5026b3
Post-Processor: gcr.io/cloud-devrel-public-resources/owlbot-python:latest@sha256:0e18b9475fbeb12d9ad4302283171edebb6baf2dfca1bd215ee3b34ed79d95d7

Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>
* fix: raise ValueError if date is out-of-bounds

* unify _datetime return type

* add relevant unit test
chalmerlowe and others added 24 commits April 17, 2025 11:08
* Update Python version to 3.13 in compliance workflow

Updates the Python version used in the matrix strategy for both the `compliance` and `compliance-prerelease` jobs in the `.github/workflows/compliance.yml` file from 3.12 to 3.13.

* Update .github/workflows/compliance.yml

* Update .github/workflows/compliance.yml

---------

Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com>
…straints, .github (AI experiment) (#336)

* feat: Add Python 3.13 updates for repo settings and Kokoro

Includes additional updates for Python 3.13 support:

- Adds 'unit (3.13)' to the required status checks in the repo settings file (`.github/sync-repo-settings.yaml`).
- Populates and updates Kokoro sample configuration files in `.kokoro/samples/python3.13/` to mirror the 3.12 setup, adjusting version-specific variables.

* 🦉 Updates from OwlBot post-processor

See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md

---------

Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com>
Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>
)

* deps: Updates deprecation warning to FutureWarning re: 3.7 and 3.8

* Changes pytest flag to ignore FutureWarning re: 3.7 and 3.8

* 🦉 Updates from OwlBot post-processor

See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md

* Changes pytest flag to ignore FutureWarning re: 3.7 and 3.8

---------

Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>
* feat: Drop support for Python 3.7 and 3.8

Removes support for Python 3.7 and 3.8, establishing Python 3.9 as the new minimum supported version.

This change involves:
- Updating `python_requires` and classifiers in `setup.py`.
- Modifying Python versions in `noxfile.py` (default, unit tests, system tests) and ensuring constraint file logic remains correct.
- Updating the GitHub Actions workflow (`unittest.yml`) matrix, runner, and coverage job version.
- Deleting constraint files for Python 3.7 and 3.8 (`testing/constraints-3.7.txt`, `testing/constraints-3.8.txt`).
- Removing Kokoro sample configuration directories (`.kokoro/samples/python3.7/`, `.kokoro/samples/python3.8/`).
- Updating supported version mentions in `README.rst`.
- Removing 3.7 and 3.8 from the `ALL_VERSIONS` list in `samples/snippets/noxfile.py`.

* Updates python version in lint.yml

* Updates owlbot, removing reference to 3.8

* Updates CONTRIBUTING.rst

* updates pytest warnings

* Removes test_samples-impl ref to older virtualenv package

* Removes references to pandas older than 1.5.0

* Removes pandas older than 1.5 and misc changes

* updates pandas in setup.py

* more updates related to pandas

* still broken

* Updates FutureWarning tests to account for unittest coverage

* Updates json array type tests to account for unittest coverage

* updates python version checks to ensure coverage

* update json test for unittest coverage

* Update pandas_backports unittests to ensure coverage

* Updates per review comments

* moves class from version specific compliance file to generic file

* Removes weird cut and paste error

* fix linting errors

* updates import statement to ensure import of JSONArrowType

* Revise required github status checks

* update linting

* temporarily marking a class as no cover

* more updates

* marked several snippets as pragma no cover

* updates linting

* Updates constraints and setup.py

* migrates class from one time compliance file to another

* updating pyarrow version

* Updates linting

* removes determine all and module reload tests

* updates re: ndarrybackedextensionarray

* testing blacken as part of owlbot processing using 3.8

* 🦉 Updates from OwlBot post-processor

See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md

* updates blacken to 3.10

* update python version in lint.yml

* 🦉 Updates from OwlBot post-processor

See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md

* updates owlbot.py

* 🦉 Updates from OwlBot post-processor

See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md

* updates owlbot.py

* testing lint.yml

* testing linting issue

* testing linting issue

* 🦉 Updates from OwlBot post-processor

See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md

* updates lint_setup session

* Update noxfile.py

---------

Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com>
Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>
Co-authored-by: release-please[bot] <55107282+release-please[bot]@users.noreply.github.com>
* test: updates tests with regex matches

* 🦉 Updates from OwlBot post-processor

See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md

* updates to np.array processing

* 🦉 Updates from OwlBot post-processor

See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md

* update imports

* update docstrings

* update docstrings

---------

Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>
* chore(deps): update all dependencies

* updates to prevent changes to docs and lint.ymls

* updates lint python version to 3.10

* remove comments, update lint python version

* 🦉 Updates from OwlBot post-processor

See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md

---------

Co-authored-by: Mend Renovate <bot@renovateapp.com>
Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>
* chore: ignore docs.yml and lint.yml in renovate

This change updates renovate.json to prevent renovate-bot from updating the specified workflow files.

* 🦉 Updates from OwlBot post-processor

See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md

* chore: exclude renovate.json from owlbot updates

This ensures that owlbot does not overwrite the changes made to `renovate.json`.

* Update owlbot.py

* Update owlbot.py

---------

Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com>
Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>
* deps: adds limits to numpy installs under 3.10 for dataproc

* tweaks some version numbers as a test
* chore(deps): update all dependencies

* Update docs.yml

* Update lint.yml

---------

Co-authored-by: Lingqing Gan <lingqing.gan@gmail.com>
🤖 I have created a release *beep* *boop*
---


##
[1.4.4](googleapis/python-db-dtypes-pandas@v1.4.3...v1.4.4)
(2025-09-08)


### Dependencies

* Adds limits to numpy installs under 3.10 for dataproc
([#370](googleapis/python-db-dtypes-pandas#370))
([6d79280](googleapis/python-db-dtypes-pandas@6d79280))

---
This PR was generated with [Release
Please](https://github.com/googleapis/release-please). See
[documentation](https://github.com/googleapis/release-please#release-please).

Co-authored-by: release-please[bot] <55107282+release-please[bot]@users.noreply.github.com>
This adds an additional trigger to the unit tests to run post-submit as
well
This PR adds Python 3.14 as a supported version, and fixed tests for the
new version

This PR also adds a dependency bound to pandas, as the [upcoming 3.0.0
version is
incompatible](https://github.com/googleapis/python-db-dtypes-pandas/issues/381)
PR created by the Librarian CLI to initialize a release. Merging this PR
will auto trigger a release.

Librarian Version: v0.7.0
Language Image:
us-central1-docker.pkg.dev/cloud-sdk-librarian-prod/images-prod/python-librarian-generator@sha256:c8612d3fffb3f6a32353b2d1abd16b61e87811866f7ec9d65b59b02eb452a620
<details><summary>db-dtypes: 1.5.0</summary>

##
[1.5.0](googleapis/python-db-dtypes-pandas@v1.4.3...v1.5.0)
(2025-12-15)

### Features

* Add support for Python 3.14 (#380)
([c164a47b](googleapis/python-db-dtypes-pandas@c164a47b))

</details>
This PR migrates admin ownership form the api-bigquery team to the python language team.
`docs/samples` is unused as the samples were removed in
googleapis/python-db-dtypes-pandas#389
…as/main' into migration.Python-db-dtypes-pandas.migration.2026-03-02_20-30-38.migrate
@parthea parthea changed the title chore(migration): Migrate code from googleapis/Python-db-dtypes-pandas into packages/db-dtypes chore(migration): Migrate code from googleapis/python-db-dtypes-pandas into packages/db-dtypes Mar 2, 2026
@google-cla
Copy link

google-cla bot commented Mar 2, 2026

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

@gemini-code-assist
Copy link
Contributor

Summary of Changes

Hello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request focuses on migrating the db-dtypes library into the main repository, setting up essential configurations for code quality, contribution guidelines, issue handling, and automated testing. It ensures the project is well-structured, maintainable, and adheres to Google's open-source standards.

Highlights

  • Code Migration: Migrated the db-dtypes code from the googleapis/Python-db-dtypes-pandas repository into the packages/db-dtypes directory within the main repository.
  • Git History Preservation: Ensured that the git history is preserved during the migration by recommending a merge-commit strategy.
  • New File Additions: Added numerous configuration files, including .coveragerc, .flake8, CODEOWNERS, CONTRIBUTING.md, issue templates, pull request templates, auto-label configurations, blunderbuss settings, header checker linting, and various workflow configurations for compliance, docs, linting, and unit testing.
  • Kokoro Build Configuration: Integrated Kokoro build configurations for continuous integration, pre-submit checks, and sample testing across multiple Python versions.
Activity
  • The primary activity is the migration of the db-dtypes library from a separate repository into the main repository.
  • Addition of various configuration files to support code quality, issue handling, and automated testing.
  • Configuration of Kokoro build system for continuous integration and testing.
Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment @gemini-code-assist Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution.

Copy link
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request migrates the db-dtypes package into this monorepo. The changes include source code, documentation, and a large number of configuration files for CI/CD and development tooling.

Overall, the migration looks solid. I've found a couple of minor issues:

  • The CONTRIBUTING.rst file contains outdated information about supported Python versions.
  • The extract_runtime_version helper function in _versions_helpers.py could be written more concisely.

I've left specific comments with suggestions for these points. After addressing them, this PR should be good to go.

Comment on lines +24 to +25
- The feature must work fully on the following CPython versions:
3.8, 3.9, 3.10, 3.11, 3.12, 3.13 and 3.14 on both UNIX and Windows.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The supported Python versions listed in this file are inconsistent with the project's configuration (setup.py requires >=3.9). Please update the file to reflect that support starts from Python 3.9.

Specifically:

  • This line should list versions from 3.9 onwards.
  • The list of supported versions around line 224 should not include 3.7 and 3.8.
  • The statement on line 248 should mention that support begins with version 3.9.
Suggested change
- The feature must work fully on the following CPython versions:
3.8, 3.9, 3.10, 3.11, 3.12, 3.13 and 3.14 on both UNIX and Windows.
- The feature must work fully on the following CPython versions:
3.9, 3.10, 3.11, 3.12, 3.13 and 3.14 on both UNIX and Windows.

Comment on lines +22 to +32
def extract_runtime_version() -> Tuple[int, int, int]:
# Retrieve the version information
version_info = sys.version_info

# Extract the major, minor, and micro components
major = version_info.major
minor = version_info.minor
micro = version_info.micro

# Display the version number in a clear format
return major, minor, micro
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

This function can be implemented more concisely by directly returning the components from sys.version_info. This improves readability and reduces verbosity. The docstring has also been updated to be more descriptive.

Suggested change
def extract_runtime_version() -> Tuple[int, int, int]:
# Retrieve the version information
version_info = sys.version_info
# Extract the major, minor, and micro components
major = version_info.major
minor = version_info.minor
micro = version_info.micro
# Display the version number in a clear format
return major, minor, micro
def extract_runtime_version() -> Tuple[int, int, int]:
"""Extracts the runtime Python version as a tuple."""
return sys.version_info.major, sys.version_info.minor, sys.version_info.micro

@parthea parthea added the do not merge Indicates a pull request not ready for merge, due to either quality or timing. label Mar 2, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

do not merge Indicates a pull request not ready for merge, due to either quality or timing.

Projects

None yet

Development

Successfully merging this pull request may close these issues.