Skip to content

Update Snowflake JDBC driver to 4.3.4 - #67

Closed
vishwasvaidya-cloudsufi wants to merge 35 commits into
data-integrations:developfrom
cloudsufi:change-snowflake-version-4.3.4
Closed

vishwasvaidya-cloudsufi wants to merge 35 commits into
data-integrations:developfrom
cloudsufi:change-snowflake-version-4.3.4

Conversation

@vishwasvaidya-cloudsufi

Copy link
Copy Markdown

Summary

Bumps snowflake-jdbc.version from 4.0.2 to 4.3.4.

Driver 4.0.2 fails to create the S3 client when the stage endpoint has no URI scheme. Snowflake Government accounts return their endpoint this way, so every read (downloadStream) fails on those accounts with:

java.lang.NullPointerException: The URI scheme of endpointOverride must not be null.

Snowflake fixed this upstream in snowflakedb/snowflake-jdbc#2572 (SNOW-3344317), first released in 4.2.0. In 4.3.4, an endpoint without a scheme gets https:// added before the client is built.

The plugin code needs no changes: the driver APIs it uses (SnowflakeConnection, UploadStreamConfig, DownloadStreamConfig, SnowflakeBasicDataSource) are the same in 4.3.4. The driver is still built for Java 8, and the CDAP parent range is unchanged, so CDAP 6.10 and 6.11 stay supported.

Changes

  • pom.xml: snowflake-jdbc.version 4.0.2 → 4.3.4
  • New SnowflakeS3StageEndpointTest: builds the S3 client the same way downloadStream does (StageInfo → StorageClientFactory.createClient) and checks the endpoint it ends up with:
    • endpoint without a scheme → https://<endpoint>
    • https:// endpoint → unchanged
    • no endpoint → no override

Testing

Driver SnowflakeS3StageEndpointTest
4.0.2 scheme-less case fails with the NPE above; other 2 pass
4.3.4 3/3 pass

MrRahulSharma and others added 30 commits November 30, 2022 22:15
[PLUGIN-1682] [cherrypicked]Fixed null output schema in case of macros for connection properties
Bumping the plugin version from 1.1.1-SNAPSHOT to 1.1.1 .
…-to-1.1.2-SNAPSHOT

bump up from 1.1.1 to 1.1.2-SNAPSHOT
added support for timestamp_tz datatype.
…estamp-fix

[cherrypick] [PLUGIN-1721] Added support for timestamp_tz datatype.
Upgrading the driver version from 3.13.24 to 3.14.4

Added fix for oauth
…th-3.14.4

[Cherrypick][PLUGIN-1731][PLUGIN-1734] Fix Oauth issue and driver upgrade to 3.14.4
…p-1.1.2

bump up from 1.1.2-SNAPSHOT to 1.1.2
…p-to-1.1.3-SNAPSHOT

bump up to 1.1.3-SNAPSHOT
Added unit test cases flow for build and fixes junits
…a-release

[cherrypick][PLUGIN-1809]Fix added to override schema
[cherrypick][PLUGIN-1816] Added fix for decimal issue not having rounding mode and made escape character configurable.
…APSHOT

Bump version to 1.11.5-SNAPSHOT for Snowflake Plugins
…LUGIN-1896: Add Get Schema button to Named Table option
…y-pick/9d8bc2caae4d6950b51d9ca0995fdde572807053

[🍒][Plugin-1883] SnowFlake Plugin - Fetch schema using Named Table
psainics and others added 5 commits August 19, 2025 13:57
…-pick-610/b295b5b70b5d9f6d47c94c3dcba82f9e12505605

[🍒][6.10] CDAP OSS migration
Bumps `snowflake-jdbc.version` from `3.14.4` to `4.0.2`.
@google-cla

google-cla Bot commented Oct 15, 2026

Copy link
Copy Markdown

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces the ability to retrieve schemas using a named table in the Snowflake Batch Source, alongside updating the Snowflake JDBC driver to version 4.3.4 and adapting the codebase to its API changes. The review feedback highlights several critical issues: a constructor initialization bug in SnowflakeBatchSourceConfig that defaults the query type and ignores the user's selection, a case-mismatch between the UI and backend enum values, a potential IllegalStateException when calling getFields() on a non-RECORD schema, and a swallowed SQLException that hinders debugging.

this.maxSplitSize = maxSplitSize;
this.schema = schema;
this.tableName = tableName;
this.importQueryType = getImportQueryType();

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

critical

In the constructor, this.importQueryType is assigned the result of getImportQueryType(). However, at this point, the field this.importQueryType has not been initialized yet (it is null), so getImportQueryType() will always return the default value (IMPORT_QUERY), completely ignoring the constructor parameter importQueryType. This will break the 'Named Table' feature as the configuration will always default to IMPORT_QUERY. Please assign the constructor parameter directly to the field.

Suggested change
this.importQueryType = getImportQueryType();
this.importQueryType = importQueryType;


@Nullable
public String getImportQueryType() {
return importQueryType == null ? ImportQueryType.IMPORT_QUERY.name() : importQueryType;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

The method getImportQueryType() returns ImportQueryType.IMPORT_QUERY.name() (which is "IMPORT_QUERY") when importQueryType is null. However, the widget configuration in Snowflake-batchsource.json defines the option ID as "importQuery" (lowercase). Returning "IMPORT_QUERY" will cause a mismatch in the UI. It should return ImportQueryType.IMPORT_QUERY.getValue() instead.

Suggested change
return importQueryType == null ? ImportQueryType.IMPORT_QUERY.name() : importQueryType;
return importQueryType == null ? ImportQueryType.IMPORT_QUERY.getValue() : importQueryType;

Comment on lines +135 to +136
boolean isImportQuerySelected = ImportQueryType.IMPORT_QUERY.getValue().equals(importQueryType);
boolean isTableNameSelected = ImportQueryType.TABLE_NAME.getValue().equals(importQueryType);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

Using direct string comparison between ImportQueryType.IMPORT_QUERY.getValue() (which is "importQuery") and the raw importQueryType field is error-prone due to potential case mismatches (e.g., "IMPORT_QUERY" vs "importQuery"). Instead, you should use the ImportQueryType.fromString(...) helper method to parse the query type and perform enum comparisons.

Suggested change
boolean isImportQuerySelected = ImportQueryType.IMPORT_QUERY.getValue().equals(importQueryType);
boolean isTableNameSelected = ImportQueryType.TABLE_NAME.getValue().equals(importQueryType);
ImportQueryType queryType = ImportQueryType.fromString(importQueryType);
boolean isImportQuerySelected = queryType == ImportQueryType.IMPORT_QUERY;
boolean isTableNameSelected = queryType == ImportQueryType.TABLE_NAME;

}
}
} catch (SQLException e) {
throw new IOException();

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

The caught SQLException is swallowed, and a generic IOException is thrown without a message or the root cause. This makes troubleshooting database connection or query issues extremely difficult. Please pass the original exception as the cause to the IOException.

Suggested change
throw new IOException();
throw new IOException("Failed to retrieve field descriptors for table: " + tableName, e);

if (Strings.isNullOrEmpty(importQuery)) {
String tableName = config.getTableName();
Schema schema = SchemaHelper.getParsedSchema(config.getSchema());
if (schema != null && schema.getFields() != null && !schema.getFields().isEmpty()) {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

If the configured schema is not of type RECORD, calling schema.getFields() will throw an IllegalStateException. To prevent this, you should verify that the schema type is RECORD before accessing its fields.

Suggested change
if (schema != null && schema.getFields() != null && !schema.getFields().isEmpty()) {
if (schema != null && schema.getType() == Schema.Type.RECORD && schema.getFields() != null && !schema.getFields().isEmpty()) {

@vishwasvaidya-cloudsufi

Copy link
Copy Markdown
Author

needs some changes

@vishwasvaidya-cloudsufi
vishwasvaidya-cloudsufi deleted the change-snowflake-version-4.3.4 branch October 15, 2026 06:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

build build

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants