# processingCsvFiles
State Resource
This is a State Resource as provided by the tymly-etl-plugin plugin.
# Purpose
Smash CSV files into more manageable files based on column values - just a thin wrapper over Smithereens (opens new window)
# Usage
# Example
{
"processingCsvFiles": {
"options": {
"parser": {
"quote": "\"",
"delimiter": ",",
"newline": "\r\n",
"skipFirstLine": false,
"trimWhitespace": true
},
"dirSplits": [
{
"columnIndex": 3,
"valueToDirMap": {
"c": "children",
"a": "adults"
}
}
],
"fileSplits": {
"columnIndex": 4,
"valueToFileMap": {
"u": {
"filename": "changes",
"outputColumns": [
{
"name": "person_no",
"columnIndex": 0
},
{
"name": "first_name",
"columnIndex": 1
},
{
"name": "last_name",
"columnIndex": 2
}
]
},
"d": {
"filename": "deletes",
"outputColumns": [
{
"name": "person_no",
"columnIndex": 0
}
]
}
}
}
},
"events": {
"csvFilesProcessed": {
"to": "importingCsvFiles"
}
}
}
}
# Options
# Optional properties
# parser
Object controlling how CSV lines should be parsed. Uses csv-streamify (opens new window) under the bonnet... with the addition of a skipFirstLine property (boolean) for conditionally ignoring header lines.
- Type:
object
# dirSplits
Used to split CSV files across different directories, depending on column values. Please see the dirSplit (opens new window) object in Smithereens (opens new window)for more information.
- Type:
array
# fileSplits
In a similar way, CSV lines can be split across different file too. Please see the fileSplit (opens new window) object in Smithereens (opens new window)for more information.
- Type:
object