added cd to upload to conda

[utkarshgupta95]

Closes #1

[aidanheerdegen]

LGTM

[aidanheerdegen]

There seems to be a problem with later versions of python.

This string

https://github.com/ACCESS-NRI/ncigrafana/blob/master/ncigrafana/UsageDataset.py#L462-L469

when it is formatted in the query here

https://github.com/ACCESS-NRI/ncigrafana/blob/master/ncigrafana/UsageDataset.py#L462-L469

ends up looking like this:

'SELECT printf("%s (%s)", Users.fullname, Users.user) as Name, scandate as Date, SUM((\'size\',)) AS totsize \n        FROM (\'UserStorage\',)\n        LEFT JOIN Users ON (\'UserStorage\',).user_id = Users.id\n        WHERE scandate between \'1984-07-01\' AND \'1984-09-30\'\n        AND project_id = (1,)\n        AND storagepoint_id = (4,)\n        GROUP BY Name, Date\n        ORDER BY Date'

so the formatting has treated all the strings and numbers as arrays.

Very odd, but I assume this is something that change in python from v3.8 to 3.9, as the former was fine. We should, in any case, test with much more modern versions (3.11 and 3.12), but I assume they also exhibit the same behaviour.

Is that enough to go on with @utkarshgupta95?

[aidanheerdegen]

BTW I debugged this by cloning the repo and running

python -m pytest --pdb -s test

I also added

        import pdb; pdb.set_trace()

In the getstorage function so I could step through the code and print out the variables and see what it was doing.

[utkarshgupta95]

@aidanheerdegen The issue seems to be with pandas >= 2.2.0. After debugging for hours I did the very basic step to print the exception for this line, which gave

'Connection' object has no attribute 'cursor'

The issue is similar to the one below, although it uses SQLAlchemy instead of dataset
https://stackoverflow.com/questions/38332787/pandas-to-sql-to-sqlite-returns-engine-object-has-no-attribute-cursor

I cannot find a similar approach for dataset Do you have idea on how we can adapt the above solution for dataset package?

Alternatively I can execute the query with dataset.query() and then convert the result obtained to pandas.

[aidanheerdegen]

I cannot find a similar approach for dataset Do you have idea on how we can adapt the above solution for dataset package?

No.

Alternatively I can execute the query with dataset.query() and then convert the result obtained to pandas.

That sounds like a good approach.

[aidanheerdegen]

Looks great! Thanks for tracking that down that error and fixing it.