Skip to content

Astyanax over Java Driver

Jump to bottom
opuneet edited this page Jan 8, 2014 · 15 revisions

Highly recommended readings

Note - this is a BETA release only

Java-Driver does some of the heavy lifting like load balancing and connection pooling and also provides some neat features such as cursor support and async requests. But note that thrift and CQL3 have several differences.

Building and supporting a unified API that straddles both drivers (thrift and Java Driver) is challenging and sometimes not feasible. Hence please review your use case to see if it is supported by this driver.

We have listed the caveats below along with several examples with the different types of schemas that are supported by the implementation.

How to select the right release / code branch

There are 2 main code branches and release versions for Astyanax - master and beta-java-driver

  • master is the current main development branch and only supports the thrift based driver
  • beta-java-driver is the branch which uses Java Driver. It also has all the code for the thrift based driver and hence is a superset of the master branch.

Any updates to the master branch (thrift based changes) will be ported forward to the beta-java-driver branch. The short term goal is to make it explicitly clear to consumers what code they are using and the longer term goal is to have just one master branch (if possible) that has both drivers supported.

Astyanax also has 2 library versions that correspond to the 2 branches

  • astyanax-1.x.x which comes from master
  • astyanax-2.0.x-beta which comes from beta-java-driver

Start with setting up your keyspace context

Example with bare minimum config

		final String clusterName = "myCluster";
		final String keyspaceName = "myKeyspace";
		
		final String SEEDS = "localhost";
		final Supplier<List<Host>> HostSupplier = new Supplier<List<Host>>() {

			@Override
			public List<Host> get() {
				Host host = new Host(SEEDS, 9160);
				return Collections.singletonList(host);
			}
		};

		AstyanaxContext<Keyspace> context = new AstyanaxContext.Builder()
		.forCluster(clusterName)
		.forKeyspace(keyspaceName)
		.withHostSupplier(HostSupplier)
		.withAstyanaxConfiguration(
				new AstyanaxConfigurationImpl()
				.setDiscoveryType(NodeDiscoveryType.DISCOVERY_SERVICE)
				.setDiscoveryDelayInSeconds(60000)
		)
		.withConnectionPoolConfiguration(
				new JavaDriverConfigBuilder()
				.build())
		.buildKeyspace(CqlFamilyFactory.getInstance());
		
		Keyspace keyspace = context.getClient();

Note the critical config that was used in the example above. Ensure that at a bare minimum, you have the following specified

  • Cluster and Keyspace name
  • HostSupplier object along with NodeDiscoveryType telling Astyanax to use the HostSupplier
  • JavaDriverConfigBuilder for the basic default config for Java Driver (Note that Astyanax does not use any of it's own defaults. It will use the Java Driver defaults directly)
  • Lastly CqlFamilyFactory telling Astyanax to switch to the astyanax-cql impl for the driver underneath

Other examples when configuring Java Driver

Below are some examples for configuring specific components of Java Driver. Note that these knobs are different from the knobs that Astyanax provides in general, but this is everything that is currently available with Java Driver. The JavaDriverConfigBuilder is your friend when you want to explicitly configure something with the Java Driver. It exposes all the config that is available with Java Driver and provides a helpful builder style fluent interface that is consistent with the rest of the Astyanax API.

Configuring Load Balancing Policy

.withConnectionPoolConfiguration(
				new JavaDriverConfigBuilder()
				.withLoadBalancingPolicy(new TokenAwarePolicy(new RoundRobinPolicy()))
				.build())

Note that the default is Round Robin. Other policies available are

  • DCAwareRoundRobinPolicy
  • LatencyAwarePolicy
  • TokenAwarePolicy

Configuring Reconnection Policy

.withConnectionPoolConfiguration(
				new JavaDriverConfigBuilder()
				.withReconnectionPolicy(new ConstantReconnectionPolicy(1000))
				.build())

All policies available are

  • ExponentialReconnectionPolicy
  • ConstantReconnectionPolicy Default is ExponentialReconnectionPolicy (base delay - 1 sec, max delay - 10 mins).

Configuring Retry Policy

.withConnectionPoolConfiguration(
				new JavaDriverConfigBuilder()
				.withRetryPolicy(DefaultRetryPolicy.INSTANCE)
				.build())

All policies available are

  • DefaultRetryPolicy
  • DowngradingConsistencyRetryPolicy
  • FallThroughRetryPolicy
  • LoggingRetryPolicy

Default is DefaultRetryPolicy (base delay - 1 sec, max delay - 10 mins).

Configuring native protocol connection port

.withConnectionPoolConfiguration(
				new JavaDriverConfigBuilder()
				.withPort(7012)
				.build())

Default is 9042

Configuring connections

.withConnectionPoolConfiguration(
			new JavaDriverConfigBuilder()
				.withCoreConnsPerHost(HostDistance.LOCAL, 3)
				.withMaxConnsPerHost(HostDistance.LOCAL, 8)
				.withMinRequestsPerConnection(HostDistance.LOCAL, 128)
				.withMaxRequestsPerConnection(HostDistance.LOCAL, 200)
				.build())

Default settings

  • min requests per conn = 25, max requests per conn = 100
  • Local - core pool size = 2, max pool size = 8
  • Remote - core pool size = 1, max pool size = 2

Configuring timeout settings

.withConnectionPoolConfiguration(
			new JavaDriverConfigBuilder()
				.withConnectTimeout(2, TimeUnit.SECONDS)
				.withReadTimeout(10, TimeUnit.SECONDS)
				.build())

Default settings

  • Connect timeout - 5 seconds
  • Read timeout - 12 seconds

Configuring jmx metrics

.withConnectionPoolConfiguration(
				new JavaDriverConfigBuilder()
				.withJmxReportingEnabled(true)
				.build())

Enabled by default

Configuring Query options

	.withConnectionPoolConfiguration(
				new JavaDriverConfigBuilder()
				.withConsistencyLevel(ConsistencyLevel.LOCAL_QUORUM)
				.withFetchSize(100)
				.build())

Default settings

  • ConsistencyLevel.ONE
  • Fetch size = 5000

Note that fetch size is the no of rows to fetch when paginating over large result sets

Cache the column family definition with your ColumnFamily object

Reads / Queries - how to use PreparedStatements for better performance

Writes / Mutations - how to use PreparedStatements for better performance

Examples

Simple ColumnFamily definition

ColumnFamily with composite columns

Regular flat table style ColumnFamily

A Netflix Original Production
Tech Blog | Twitter @NetflixOSS | Jobs

Clone this wiki locally