Stacks

A stack describes the layout of a cluster, including flavor and number of nodes and what services should be installed, e.g. HDFS, Storm, Spark, etc. The Cloud Big Data provides a number of pre-defined stacks on which you may base your clusters. To see a list of stacks, use the list() method:

>>> stacks = lava.stacks.list()
>>> stacks
[Stack(id='ALL_HDP2_2', name='HDP 2.2 with all services', description, distro, links, services),
 Stack(id='HADOOP_HDP2_2', name='Hadoop HDP 2.2', description, distro, links, services),
 Stack(id='HDP2_2_SANDBOX', name='Hadoop HDP 2.2 All-in-One Sandbox', description, distro, links, services),
 Stack(id='KAFKA_HDP2_2', name='Kafka on HDP 2.2', description, distro, links, services),
 Stack(id='STORM_HDP2_2', name='Real-time processing on HDP 2.2 with Storm/Kafka', description, distro, links, services)]

>>> all_hdp2_2 = stacks[0]
>>> all_hdp2_2.distro
'HDP2.2'

>>> all_hdp2_2.services
[StackService(name='Flume', modes),
 StackService(name='HDFS', modes),
 StackService(name='Zookeeper', modes),
 StackService(name='Sqoop', modes),
 StackService(name='MapReduce', modes),
 StackService(name='Hive', modes),
 StackService(name='Pig', modes),
 StackService(name='YARN', modes),
 StackService(name='Storm', modes),
 StackService(name='Oozie', modes),
 StackService(name='Falcon', modes),
 StackService(name='Kafka', modes)]

If you want to get more detail on a stack, either use the get() method, or call refresh() method on the stack object:

>>> all_hdp2_2 = all_hdp2_2.refresh()
>>> all_hdp2_2
StackDetail(id='ALL_HDP2_2', name='HDP 2.2 with all services', created, description, distro, links, node_groups, services)

>>> all_hdp2_2.node_groups
[StackNodeGroup(id='gateway', flavor_id, components, count, resource_limits),
 StackNodeGroup(id='master', flavor_id, components, count, resource_limits),
 StackNodeGroup(id='secondary', flavor_id, components, count, resource_limits),
 StackNodeGroup(id='service-master', flavor_id, components, count, resource_limits),
 StackNodeGroup(id='slave', flavor_id, components, count, resource_limits),
 StackNodeGroup(id='zookeeper', flavor_id, components, count, resource_limits)]

>>> all_hdp2_2.node_groups[0].components
[{'name': 'HiveServer2'},
 {'name': 'HiveMetastore'},
 {'name': 'HiveAPI'},
 {'name': 'HiveClient'},
 {'name': 'PigClient'},
 {'name': 'MRClient'},
 {'name': 'SqoopClient'},
 {'name': 'OozieClient'},
 {'name': 'FlumeHandler'},
 {'name': 'ZookeeperClient'},
 {'name': 'FalconClient'}]

Note

In the future, you will be able to create and delete your own custom stacks.

API Reference

class lavaclient.api.stacks.Resource

Flavors API methods

create(name, distro, services, node_groups=None, description=None)

Create a stack

Parameters:
  • name – Stack name
  • distro – Valid distro identifier
  • services – List of services. Each should have a name and optionally a list of modes
  • node_groups – List of node groups for the cluster
Returns:

StackDetail
delete(stack_id)

Delete a stack

Parameters:stack_id – Stack ID
get(stack_id)

Get a specific stack

Parameters:stack_id – Stack ID
Returns:StackDetail
list()

List all stacks

Returns:List of Stack objects
class lavaclient.api.response.Stack

Same as StackDetail, except missing created and node_groups attributes

class lavaclient.api.response.StackDetail
delete()

Delete this stack.

describe()

Return a pretty-formatted string that describes the format of the data

refresh()

Refresh this stack. If this object was returned from list(), it will be converted into StackDetail

Returns:StackDetail
to_dict()

Convert the config to a plain python dictionary

created

datetime corresponding to creation date

description
distro

Distribution ID

id
name
node_groups

See: StackNodeGroup

services

See: StackService

class lavaclient.api.response.StackService
describe()

Return a pretty-formatted string that describes the format of the data

to_dict()

Convert the config to a plain python dictionary

modes
name
class lavaclient.api.response.StackNodeGroup
describe()

Return a pretty-formatted string that describes the format of the data

to_dict()

Convert the config to a plain python dictionary

components
count
flavor_id
id
resource_limits

See: ResourceLimits

class lavaclient.api.response.ResourceLimits
describe()

Return a pretty-formatted string that describes the format of the data

to_dict()

Convert the config to a plain python dictionary

max_count
min_count
min_ram