Skip to content

[fix](paimon) adapt FE for paimon 1.3.1 and iceberg 1.10.1#60876

Open
xylaaaaa wants to merge 4 commits intoapache:masterfrom
xylaaaaa:fix/hive-catalog-shade-version-paimon-snapshot-master
Open

[fix](paimon) adapt FE for paimon 1.3.1 and iceberg 1.10.1#60876
xylaaaaa wants to merge 4 commits intoapache:masterfrom
xylaaaaa:fix/hive-catalog-shade-version-paimon-snapshot-master

Conversation

@xylaaaaa
Copy link
Contributor

@xylaaaaa xylaaaaa commented Feb 27, 2026

Summary

  • bump doris.hive.catalog.shade.version to 3.1.0-PAIMON_JUNWEI_SNAPSHOT
  • bump iceberg.version to 1.10.1
  • bump paimon.version to 1.3.1
  • adapt paimon metrics API changes in FE:
  • MetricRegistry is now an interface (use implements)
  • withMetricsRegistry renamed to withMetricRegistry

Verification

  • cd fe && /mnt/disk2/yunyou/chenjunwei/doris_tools/apache-maven-3.9.9/bin/mvn -DskipTests -pl fe-core -am compile
  • build result: SUCCESS

Copilot AI review requested due to automatic review settings February 27, 2026 06:46
@hello-stephen
Copy link
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR updates Doris FE’s Paimon/Iceberg integration to match upstream dependency/API changes (Paimon 1.3.1 + Iceberg 1.10.1), including adapting the FE-side metrics registry wiring.

Changes:

  • Bump FE-managed dependency versions for Iceberg and Paimon.
  • Update Paimon scan code to use the renamed metrics-registration API.
  • Refactor PaimonMetricRegistry to match Paimon’s updated MetricRegistry type (now an interface).

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.

File Description
fe/pom.xml Updates Iceberg/Paimon versions and changes the hive-catalog-shade version property.
fe/fe-core/src/main/java/org/apache/doris/datasource/paimon/source/PaimonScanNode.java Adjusts Paimon scan metrics registration call to the new API name.
fe/fe-core/src/main/java/org/apache/doris/datasource/paimon/profile/PaimonMetricRegistry.java Updates registry implementation to implement Paimon’s MetricRegistry interface and exposes the required method.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

<properties>
<doris.hive.catalog.shade.version>3.1.0</doris.hive.catalog.shade.version>
<doris.hive.catalog.shade.version>3.1.0-PAIMON_JUNWEI_SNAPSHOT</doris.hive.catalog.shade.version>
<!-- iceberg 1.9.1 depends avro on 1.12 -->
Copy link

Copilot AI Feb 27, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This comment references Iceberg 1.9.1, but iceberg.version is now 1.10.1. Please update the comment to avoid confusion (or make it version-agnostic, e.g., ‘Iceberg depends on Avro 1.12.x’ if that’s the intended constraint).

Suggested change
<!-- iceberg 1.9.1 depends avro on 1.12 -->
<!-- Iceberg depends on Avro 1.12.x -->

Copilot uses AI. Check for mistakes.
fe/pom.xml Outdated
</modules>
<properties>
<doris.hive.catalog.shade.version>3.1.0</doris.hive.catalog.shade.version>
<doris.hive.catalog.shade.version>3.1.0-PAIMON_JUNWEI_SNAPSHOT</doris.hive.catalog.shade.version>
Copy link

Copilot AI Feb 27, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The doris.hive.catalog.shade.version is set to a developer-specific snapshot (3.1.0-PAIMON_JUNWEI_SNAPSHOT). Unless this artifact/version is published to the configured Maven repositories, this will make the FE build non-reproducible and likely fail for other developers/CI. Please switch to a released/shared version (or add the proper repository and use a team-standard version naming).

Suggested change
<doris.hive.catalog.shade.version>3.1.0-PAIMON_JUNWEI_SNAPSHOT</doris.hive.catalog.shade.version>
<doris.hive.catalog.shade.version>3.1.0</doris.hive.catalog.shade.version>

Copilot uses AI. Check for mistakes.
@CalvinKirs
Copy link
Member

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 28582 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit b05aeae304bd1a935d6514a4b1d0f5c0e626e4b0, data reload: false

------ Round 1 ----------------------------------
============================================
q1	17632	4506	4314	4314
q2	q3	10650	768	522	522
q4	4679	360	260	260
q5	7544	1184	1003	1003
q6	174	185	146	146
q7	756	830	679	679
q8	9305	1422	1303	1303
q9	4973	4671	4643	4643
q10	6813	1877	1636	1636
q11	461	259	244	244
q12	718	563	463	463
q13	17768	4206	3423	3423
q14	224	230	218	218
q15	955	791	790	790
q16	736	719	669	669
q17	714	902	434	434
q18	6023	5368	5217	5217
q19	1266	967	610	610
q20	497	491	395	395
q21	4465	1823	1376	1376
q22	349	288	237	237
Total cold run time: 96702 ms
Total hot run time: 28582 ms

----- Round 2, with runtime_filter_mode=off -----
============================================
q1	4423	4328	4350	4328
q2	q3	1758	2178	1715	1715
q4	829	1151	760	760
q5	4016	4283	4325	4283
q6	177	172	139	139
q7	1730	1590	1492	1492
q8	2383	2643	2455	2455
q9	7293	7522	7323	7323
q10	2664	2810	2447	2447
q11	531	449	419	419
q12	505	651	482	482
q13	3942	4424	3561	3561
q14	279	296	273	273
q15	876	865	835	835
q16	767	750	773	750
q17	1189	1533	1342	1342
q18	7209	6909	6641	6641
q19	902	873	911	873
q20	2071	2176	2022	2022
q21	4080	3758	3391	3391
q22	448	434	392	392
Total cold run time: 48072 ms
Total hot run time: 45923 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 183329 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit b05aeae304bd1a935d6514a4b1d0f5c0e626e4b0, data reload: false

query5	4823	638	523	523
query6	327	213	204	204
query7	4211	461	264	264
query8	331	239	240	239
query9	8739	2791	2754	2754
query10	519	373	352	352
query11	16968	17397	17057	17057
query12	204	133	131	131
query13	1273	496	363	363
query14	7000	3481	3025	3025
query14_1	2909	2908	2882	2882
query15	211	198	178	178
query16	1025	496	518	496
query17	1340	765	645	645
query18	3002	474	376	376
query19	231	226	227	226
query20	181	164	137	137
query21	214	157	128	128
query22	5602	5223	4792	4792
query23	17215	16758	16542	16542
query23_1	16672	16704	16687	16687
query24	7191	1648	1227	1227
query24_1	1238	1234	1217	1217
query25	560	477	423	423
query26	1230	266	152	152
query27	2726	463	286	286
query28	4465	1876	1885	1876
query29	802	556	473	473
query30	311	246	212	212
query31	873	711	639	639
query32	81	69	71	69
query33	520	328	275	275
query34	914	894	558	558
query35	647	658	609	609
query36	1044	1136	959	959
query37	125	97	80	80
query38	2943	2910	2862	2862
query39	912	886	861	861
query39_1	854	812	832	812
query40	237	149	134	134
query41	64	61	57	57
query42	103	103	101	101
query43	366	375	361	361
query44	
query45	201	184	185	184
query46	876	978	598	598
query47	2093	2130	2033	2033
query48	312	326	237	237
query49	627	501	370	370
query50	693	285	218	218
query51	4103	4108	4128	4108
query52	105	106	96	96
query53	291	336	279	279
query54	294	267	260	260
query55	88	81	78	78
query56	306	307	342	307
query57	1336	1334	1275	1275
query58	286	272	265	265
query59	2644	2664	2531	2531
query60	341	328	316	316
query61	151	145	147	145
query62	639	591	547	547
query63	303	276	273	273
query64	4883	1276	1013	1013
query65	
query66	1367	448	348	348
query67	16311	16426	16211	16211
query68	
query69	414	300	286	286
query70	933	911	943	911
query71	334	300	293	293
query72	2774	2726	2373	2373
query73	538	550	323	323
query74	10014	9999	9787	9787
query75	2870	2740	2459	2459
query76	2321	1052	669	669
query77	377	370	312	312
query78	11229	11279	10762	10762
query79	3083	800	625	625
query80	1743	634	525	525
query81	595	282	250	250
query82	1015	154	114	114
query83	332	260	232	232
query84	256	116	95	95
query85	893	476	431	431
query86	490	311	290	290
query87	3080	3082	2974	2974
query88	3566	2661	2655	2655
query89	419	370	351	351
query90	2057	170	169	169
query91	161	151	137	137
query92	89	74	69	69
query93	2311	831	505	505
query94	645	315	279	279
query95	585	341	307	307
query96	623	510	228	228
query97	2461	2492	2397	2397
query98	219	217	210	210
query99	966	960	865	865
Total cold run time: 258094 ms
Total hot run time: 183329 ms

@CalvinKirs
Copy link
Member

run buildall

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants