On Thu, Mar 20, 2008 at 06:58:42AM -0600, Spencer Parker wrote:
> I am using Python to parse the file and then dumps the info into a MySQL
> database to then be displayed and aggregated for a webpage. O know I just
> need to parse every 25 places in the index...then just do the next 28 and so
> on. I'm pretty beginner at programming in general...so its nice having
> something like this to help teach me Python.
[Please CC replies to et-mgmt-list]
Attached is an example CSV file from the latest version of virt-top so
we have something in common to talk about. Load this into a
spreadsheet program like OpenOffice Calc.
The basic problem with CSV is that it's a 2D file format, and if you
have more dimensions than two in your data, as in this case, you have
to squeeze them into two somehow. This is what I'm doing here.
The data format is:
[common columns] [domain columns]
time [dom1] [dom2] [dom3] ....
|
|
V
Looking at the spreadsheet, columns A-Q are the common columns.
Column R is the first domain column. Note the title "Domain ID".
This title always marks the first domain column.
Column Y is the last domain column for the first domain. Note that it
is the last column that has a title, and this is how you detect it.
Column Z is the first domain column for the second domain. It
contains the domain ID of the second domain. The second domain goes
through columns Z - AG.
If there was a third domain it would start at column AH (which would
be its domain ID), and so on.
You can work out the width of each domain as Y-R (8 columns in this
instance, but you have to calculate it because it might be different
in future versions of virt-top, or even in current versions of
virt-top if you use the --no-csv-* command line flags).
BTW, which Python library are you using to parse CSV? You should
always use a library. You can't get away with just splitting at comma
characters. If you have the ocaml-csv module installed from Fedora
then there's a command-line tool called csvtool included which can
also do some manipulations of CSV files correctly.
Rich.
--
Richard Jones, Emerging Technologies, Red Hat http://et.redhat.com/~rjones
virt-p2v converts physical machines to virtual machines. Boot with a
live CD or over the network (PXE) and turn machines into Xen guests.
http://et.redhat.com/~rjones/virt-p2v
Hostname,Time,Arch,Physical CPUs,Count,Running,Blocked,Paused,Shutdown,Shutoff,Crashed,Active,Inactive,%CPU,Total memory (KB),Total guest memory (KB),Total CPU time (ns),Domain ID,Domain name,CPU (ns),%CPU,Block RDRQ,Block WRRQ,Net RXBY,Net TXBY
amd.home.annexia.org,13:03:50,x86_64,4,2,2,0,0,0,0,0,2,0,0.0,1024000,1024000,0,1,rhel51x32kvm,0.,0.,,,,,2,rhel51x64kvm,0.,0.,,,,
amd.home.annexia.org,13:03:53,x86_64,4,2,2,0,0,0,0,0,2,0,6.6,1024000,1024000,790000000,1,rhel51x32kvm,430000000.,3.58034253276,,,104,0,2,rhel51x64kvm,360000000.,2.99749607394,,,104,0
amd.home.annexia.org,13:03:56,x86_64,4,2,2,0,0,0,0,0,2,0,6.5,1024000,1024000,780000000,1,rhel51x32kvm,440000000.,3.66482943344,,,52,342,2,rhel51x64kvm,340000000.,2.83191365311,,,394,0
amd.home.annexia.org,13:03:59,x86_64,4,2,2,0,0,0,0,0,2,0,6.5,1024000,1024000,780000000,1,rhel51x32kvm,440000000.,3.66471706899,,,446,0,2,rhel51x64kvm,340000000.,2.83182682603,,,104,342
amd.home.annexia.org,13:04:02,x86_64,4,2,2,0,0,0,0,0,2,0,6.3,1024000,1024000,760000000,1,rhel51x32kvm,440000000.,3.66482943344,,,52,0,2,rhel51x64kvm,320000000.,2.66533049704,,,52,0
amd.home.annexia.org,13:04:05,x86_64,4,2,2,0,0,0,0,0,2,0,6.8,1024000,1024000,820000000,1,rhel51x32kvm,440000000.,3.66482797789,,,104,0,2,rhel51x64kvm,380000000.,3.16507870818,,,104,0
amd.home.annexia.org,13:04:08,x86_64,4,2,2,0,0,0,0,0,2,0,6.2,1024000,1024000,750000000,1,rhel51x32kvm,430000000.,3.57913635914,,,52,0,2,rhel51x64kvm,320000000.,2.66354333703,,,52,0
amd.home.annexia.org,13:04:11,x86_64,4,2,2,0,0,0,0,0,2,0,6.6,1024000,1024000,790000000,1,rhel51x32kvm,450000000.,3.74807337613,,,446,0,2,rhel51x64kvm,340000000.,2.83187766197,,,104,342
amd.home.annexia.org,13:04:14,x86_64,4,2,2,0,0,0,0,0,2,0,6.6,1024000,1024000,790000000,1,rhel51x32kvm,440000000.,3.6622060958,,,52,342,2,rhel51x64kvm,350000000.,2.9131184853,,,394,0
amd.home.annexia.org,13:04:17,x86_64,4,2,2,0,0,0,0,0,2,0,6.6,1024000,1024000,790000000,1,rhel51x32kvm,440000000.,3.66491473022,,,104,0,2,rhel51x64kvm,350000000.,2.91527308086,,,104,0
csvtool - Copyright (C) 2005-2006 Richard W.M. Jones, Merjis Ltd.
csvtool is a tool for performing manipulations on CSV files from shell scripts.
Summary:
csvtool [-options] command [command-args] input.csv [input2.csv [...]]
Commands:
col <column-spec>
Return one or more columns from the CSV file.
For <column-spec>, see below.
Example: csvtool col 1-3,6 input.csv > output.csv
namedcol <names>
Assuming the first row of the CSV file is a list of column headings,
this returned the column(s) with the named headings.
<names> is a comma-separated list of names.
Example: csvtool namedcol Account,Cost input.csv > output.csv
width
Print the maximum width of the CSV file (number of columns in the
widest row).
height
Print the number of rows in the CSV file.
For most CSV files this is equivalent to 'wc -l', but note that
some CSV files can contain a row which breaks over two (or more)
lines.
setcolumns cols
Set the number of columns to cols (this also makes the CSV file
square). Any short rows are padding with blank cells. Any
long rows are truncated.
setrows rows
'setrows n' sets the number of rows to 'n'. If there are fewer
than 'n' rows in the CSV files, then empty blank lines are added.
head rows
take rows
'head n' and 'take n' (which are synonyms) take the first 'n'
rows. If there are fewer than 'n' rows, padding is not added.
drop rows
Drop the first 'rows' rows and return the rest (if any).
Example:
To remove the headings from a CSV file with headings:
csvtool drop 1 input.csv > output.csv
To extract rows 11 through 20 from a file:
csvtool drop 10 input.csv | csvtool take 10 - > output.csv
cat
This concatenates the input files together and writes them to
the output. You can use this to change the separator character.
Example: csvtool -t TAB -u COMMA cat input.tsv > output.csv
join <column-spec1> <column-spec2>
Join (collate) multiple CSV files together.
<column-spec1> controls which columns are compared.
<column-spec2> controls which columns are copied into the new file.
Example:
csvtool join 1 2 coll1.csv coll2.csv > output.csv
In the above example, if coll1.csv contains:
Computers,$40
Software,$100
and coll2.csv contains:
Computers,$50
then the output will be:
Computers,$40,$50
Software,$100,
square
Make the CSV square, so all rows have the same length.
Example: csvtool square input.csv > input-square.csv
sub r c rows cols
Take a square subset of the CSV, top left at row r, column c, which
is rows deep and cols wide. 'r' and 'c' count from 1, or
from 0 if -z option is given.
replace <column-spec> update.csv original.csv
Replace rows in original.csv with rows from update.csv. The columns
in <column-spec> only are used to compare rows in input.csv and
update.csv to see if they are candidates for replacement.
Example:
csvtool replace 3 updates.csv original.csv > new.csv
mv new.csv original.csv
call command
This calls the external command (or shell function) 'command'
followed by a parameter for each column in the CSV file. The
external command is called once for each row in the CSV file.
If any command returns a non-zero exit code then the whole
program terminates.
Tip:
Use the shell command 'export -f funcname' to export
a shell function for use as a command. Within the
function, use the positional parameters $1, $2, ...
to refer to the columns.
Example (with a shell function):
function test {
echo Column 1: $1
echo Column 2: $2
}
export -f test
csvtool call test my.csv
In the above example, if my.csv contains:
how,now
brown,cow
then the output is:
Column 1: how
Column 2: now
Column 1: brown
Column 2: cow
readable
Print the input CSV in a readable format.
Column specs:
A <column-spec> is a comma-separated list of column numbers
or column ranges.
Examples:
1 Column 1 (the first, leftmost column)
2,5,7 Columns 2, 5 and 7
1-3,5 Columns 1, 2, 3 and 5
1,5- Columns 1, 5 and up.
Columns are numbered starting from 1 unless the -z option is given.
Input files:
csvtool takes a list of input file(s) from the command line.
If an input filename is '-' then take input from stdin.
Output file:
Normally the output is written to stdout. Use the -o option
to override this.
Separators:
The default separator character is , (comma). To change this
on input or output see the -t and -u options respectively.
Use -t TAB or -u TAB (literally T-A-B!) to specify tab-separated
files.
Options:
-t Input separator char. Use -t TAB for tab separated input.
-u Output separator char. Use -u TAB for tab separated output.
-o Write output to file (instead of stdout)
-z Number columns from 0 instead of 1
-
-help Display this list of options
--help Display this list of options
_______________________________________________
et-mgmt-tools mailing list
et-mgmt-tools@xxxxxxxxxx
https://www.redhat.com/mailman/listinfo/et-mgmt-tools
