Skip to content

[SPARK-60198][PYTHON] Document the transpiler hook's runtime contract - #59986

Open
guan404ming wants to merge 1 commit into
apache:masterfrom
guan404ming:SPARK-60198-transpiler-hook-contract
Open

guan404ming wants to merge 1 commit into
apache:masterfrom
guan404ming:SPARK-60198-transpiler-hook-contract

Conversation

@guan404ming

Copy link
Copy Markdown
Member

What changes were proposed in this pull request?

  • Document the transpiler hook's runtime contract in the AbstractTranspiler docstring (SPARK-60198).
  • State that _transpile_from_ast runs synchronously on the driver at UDF construction.
  • State that it runs once per input-type variant, and never under Spark Connect.
  • Mark the whole hook, this contract included, as experimental.

Why are the changes needed?

  • Custom transpilers already rely on this timing for side effects, such as registering a Java UDF.
  • Without a written contract, lazy transpilation or Connect support could break them silently.

Does this PR introduce any user-facing change?

No, this only changes a docstring.

How was this patch tested?

  • Docstring only, so no new test.
  • Checked the statements against UserDefinedFunction.__init__ and _transpile_func.

Was this patch authored or co-authored using generative AI tooling?

Generated-by: Claude Code (Opus 5.5)

@guan404ming
guan404ming marked this pull request as ready for review October 31, 2026 11:23

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

1 participant