datapad.fields

Functions for manipulating and retrieving fields within structured sequences. These functions are often used in conjunction with the datapad.Sequence.map() method to manipulate structured sequences.

Functions

add(*args)	Constructs a function to add or append new fields to a dict or list of data.
apply(*args)	Constructs a function to transform the fields in a dict or list of data.
asdict([keys])	Constructs a function that will convert a list into a dict using the given keys.
get(key[, default])	Constructs a function F(data) to get a field-value given a field-key from a list or dict.
select(keys)	Constructs a function to select fields from a list or dict.

datapad.fields.add(*args)

Constructs a function to add or append new fields to a dict or list of data. The returned function will assume one of the following signatures:

F(data: list) -> list
F(data: dict) -> dict

This function is commonly used in conjunction with datapad.Sequence.map() to add new fields to structured sequence data.

Possible ways to call the add function:

# Append a single field with a value of `func(data)` to list `data`
add(func: Function)

# Add a single field to dict `data` by applying `data[key]` = `func(data)`
add(key: Hashable, func: Function)

# Append multiple fields with values [f(data) for f in funcs] to a list `data`
add(funcs: List[func: Function])

# Add multiple fields to a dict `data` by applying `data[key]` = `funcs[key](data)`
add(funcs: Dict[key: Hashable, value: Function])

Parameters:

• key (hashable) – A string, int, or other hashable value to be used as the name of the added field.
• func (function) – A function of the form f(data) to compute the value of the newly added element.
• funcs (List[Function], or Dict[Hashable, Function]) –
  • If funcs is a list, data will be extended with values computed from f(data) for f in funcs
  • If funcs is a dict, data[key] will be updated with values computed from funcs[key]. Note: this will overwrite any existing keys with the same name in data.

Examples

In the examples below, assume data represents a single row or element from a datapad.Sequence object.

Add a single new element to end of list

>>> data = [1, 2, 3]
>>> F = add(lambda data: data[0] * data[1] * data[2])
>>> F(data)
[1, 2, 3, 6]

Add a single new dictionary key and value:

>>> data = {'a': 1, 'b': 2}
>>> F = add('c', lambda data: data['a'] + data['b'])
>>> F(data)
{'a': 1, 'b': 2, 'c': 3}

Add new dictionary keys:

>>> data = {'a': 1, 'b': 2}
>>> F = add({ 'c': (lambda data: data['a'] + data['b']), 'd': lambda data: 10})
>>> F(data)
{'a': 1, 'b': 2, 'c': 3, 'd': 10}

Append new list elements:

>>> data = ['foo', 'bar']
>>> F = add([lambda data: data[1] + data[0]])
>>> F(data)
['foo', 'bar', 'barfoo']

datapad.fields.apply(*args)

Constructs a function to transform the fields in a dict or list of data. The returned function will assume one of the following signatures:

F(data:list) -> list
F(data:dict) -> dict

This function is commonly used in conjunction with datapad.Sequence.map() to manipulate structured sequence data.

Possible ways to call the apply function:

# apply func to all fields in data
apply(func: Function)

# apply func to data[key] field
apply(key: Hashable, func: Function)

# apply funcs[i] to data[i] fields
apply(funcs: List[Function])

# apply funcs[key] to data[key] fields
apply(funcs: Dict[Hashable, Function])

Parameters:

• key (string, int, hashable) – A string, int, or other hashable value to be used to look up the field value to transform.
• func (function) – A function that will take a single field value and transform it into a new value.
• funcs (List[Function], or Dict[Hashable, Function]) –
  • If funcs is a list, F(data) will apply funcs[i] to data[i].
  • If funcs is a dict, F(data) will apply funcs[key] to data[key].
  • If a key in data has no corresponding key in funcs, return the data[key] field untransformed.
  • If a key in funcs has no corresponding key in data, abstain from applying funcs[key]

Examples

In the examples below, assume data represents a single row or element from a datapad.Sequence object.

Apply a single function to all fields:

>>> data = [1, 2, 3]
>>> F = apply(lambda x: x * 2)
>>> F(data)
[2, 4, 6]

Apply a single function to a single field of a list (i.e. the value associated with the 2nd index):

>>> data = [1, 2, 3]
>>> F = apply(1, lambda x: x * 2)
>>> F(data)
[1, 4, 3]

Apply a single function to a single field of a dict (i.e. the value associated with key ‘a’):

>>> data = {'a': 1, 'b': 2, 'c': 3}
>>> F = apply('a', lambda x: x * 2)
>>> F(data)
{'a': 2, 'b': 2, 'c': 3}

Apply multiple functions to several fields of a dict (i.e. the values associated with keys ‘a’, ‘b’, ‘c’, and ‘d’):

>>> data = {'a': 1, 'b': 2, 'c': 3, 'd': 4}
>>> F = apply({'a': lambda x: x*3, 'b': lambda x: 'foo'})
>>> F(data)
{'a': 3, 'b': 'foo', 'c': 3, 'd': 4}

Apply multiple functions to several fields of a list (i.e. the values associated with index 0, and 1):

>>> data = [1, 2, 3, 4]
>>> F = apply({1: lambda x: x*3, 0: lambda x: 'foo'})
>>> F(data)
['foo', 6, 3, 4]

Apply multiple functions to several fields of a list (i.e. the values associated with index 0, and 1):

>>> data = [1, 2, 3, 4]
>>> F = apply([lambda x: x*2, lambda x: x*3])
>>> F(data)
[2, 6, 3, 4]

datapad.fields.asdict(keys=None)

Constructs a function that will convert a list into a dict using the given keys. The returned function will have the signature:

F(data:list) -> dict

This function is commonly used in conjunction with datapad.Sequence.map() to manipulate structured sequence data.

Parameters:keys (list) – A list of strings or any other valid hashable type that will be associated with each element of the input data list. If keys is None, F(data) will use the indices of each list field as the key.

Examples

Convert list to dict using pre-defined keys:

>>> data = [1, 2, 3]
>>> F = asdict(['a', 'b', 'c'])
>>> F(data)
{'a': 1, 'b': 2, 'c': 3}

Convert list to dict using list indices as keys:

>>> data = [1, 2, 3]
>>> F = asdict()
>>> F(data)
{0: 1, 1: 2, 2: 3}

datapad.fields.get(key, default=None)

Constructs a function F(data) to get a field-value given a field-key from a list or dict. If a field-key is not present in data, return default value. This function is commonly used as a keying-function for Sequence.map, Sequence.sort, Sequence.groupby, and Sequence.join.

Examples

In the examples below, assume data is an element or row coming from a Sequence.

>>> F = get('a', 'foobar')
>>> F({'a': 1})
1
>>> F({'b': 2})
'foobar'

>>> F = get(0, "default")
>>> F([10])
10

>>> F = get(1, "default")
>>> F([10])
'default'

datapad.fields.select(keys)

Constructs a function to select fields from a list or dict. If a field-key is not present in data, default to None.

Select from a list:

>>> data = [1, 2, 3]
>>> F = select([0, 2])
>>> F(data)
[1, 3]

Select from a dictionary:

>>> data = {'a': 2, 'b': 1, 'c': 4}
>>> F = select(['c', 'b', 'k'])
>>> F(data)
{'c': 4, 'b': 1, 'k': None}