[trino] Add initial connector build and plugin skeleton - #4207
yangshangqing95 wants to merge 5 commits into
Conversation
|
@yangshangqing95, Once more, thank you for driving this. One thought here: Fluss has recently created a new organization to support its ecosystem I think the fluss-trino-connector would be a great fit for that repo initially. WDYT? |
|
Hi @polyzos, However, especially after looking through some of the previous Trino-related PRs and issues, If we first develop the connector to feature completeness in That said, I do agree with an important part of your suggestion: Ideally, I would see How we define "Mature" is also a question. I’ve also seen many promising ideas gradually lose momentum when there wasn’t a clear path to keep them moving forward. That is always a pity, and it is one of the reasons I think an incremental development model is valuable here. For me, "mature enough" would therefore be less about having every feature or optimization implemented, and more about having stable read semantics, solid integration tests, a reasonably stable public/configuration surface, and a clear compatibility and maintenance model. Even after reaching that point, there could still be plenty of incremental work to do. At the same time, I think the functional scope covered by the current design is already relatively clear, complete, and manageable. Given that scope, reaching a solid initial implementation within a release cycle appears quite achievable. This is also why I still think incremental development in the main repository has value. . As long as each change is well-scoped, independently reviewable, properly tested, and does not negatively affect the existing Fluss build or runtime, we can continue evolving it through small PRs. The isolated build environment I proposed for the Trino module is intended to preserve exactly this boundary. I also think this could be an interesting model beyond the Trino connector itself. If it works well, So I really like the idea of using I’d be very happy to discuss this further, and I’m looking forward to hearing your thoughts and continuing the discussion with the community. Best, |
fbc1d03 to
59afbff
Compare
0e0cfe6 to
c44ed03
Compare
4e04e9f to
c5c8bfb
Compare
c5c8bfb to
964eea7
Compare
This PR introduces a native, read-only Fluss Trino connector proposed in #4197.
The connector supports metadata discovery and bounded reads from partitioned and non-partitioned Log Tables and Primary Key Tables.
Changes
fluss-trinoMaven buildDESCRIBE,SHOW CREATE TABLE, and the"table$columns"system table.Scope and limitations
Log reads use fixed per-bucket offset boundaries. Primary Key Table snapshots open independently when each bucket scan starts. Neither provides a globally consistent snapshot across buckets or partitions.
Predicates, aggregations, sorting and limits are evaluated by Trino. Pushdown, writes, DDL, Lakehouse reads, lookup, Union Read and task retries are outside this PR's scope.
Client-side scanner memory is not fully exposed by the Fluss client, so memory accounting remains incomplete.
The module stays outside the default Fluss Maven reactor to accommodate Trino's JDK requirements without changing those of the existing Fluss build. Connector source follows Fluss code style and Java 8 syntax conventions; the plugin itself requires JDK 25.
For Arrow-backed reads, add the following option to Trino's
etc/jvm.configon every coordinator and worker, then restart the servers: