Generic Utils¶
These are generic utils used across the Sparkly library.
-
sparkly.utils.
absolute_path
(file_path, *rel_path)[source]¶ Returns absolute path to file.
- Usage:
>>> absolute_path('/my/current/dir/x.txt', '..', 'x.txt') '/my/current/x.txt'
>>> absolute_path('/my/current/dir/x.txt', 'relative', 'path') '/my/current/dir/relative/path'
>>> import os >>> absolute_path('x.txt', 'relative/path') == os.getcwd() + '/relative/path' True
Parameters: - file_path (str) – file
- rel_path (list[str]) – path parts
Returns: str
-
sparkly.utils.
kafka_get_topics_offsets
(host, topic, port=9092)[source]¶ Returns available partitions and their offsets for the given topic.
Parameters: - host (str) – Kafka host.
- topic (str) – Kafka topic.
- port (int) – Kafka port.
Returns: [ – [(partition, start_offset, end_offset)].
Return type: int, int, int