It’s a bit more complicated than one command. I believe you need to get to the import page, record the session id and token, and then use those with your curl command. This can be done via a script file.
@lisa@danwaterloo A few pages need to be run sequentially - login, the import page, then submit the import page.
This shell script seems to work for me importing the file import.csv into list number 9 using comma as the delimiter.
In the meantime I started updating the database directly but this shell script works perfectly and is the better option I think. I added -F overwrite=‘yes’ to update existing entries.
On the server, however, I get an invalid security token. I can login successfully via curl but in the 3rd curl command I get an error with awk. It fails to create a formpost data. In the shell it says
scripts# curl -v --cookie-jar cookie.txt -b cookie.txt -F formtoken=`awk '/formtoken/ {match($0,/name="formtoken" value="(.{32})"/,arr); print arr[1]}' <import2.log` -F lists[2]=2 -F import_file=@11_development.csv -F import_field_delimiter=',' -F overwrite='yes' -F import=import "http://mydomain.com/lists/admin/?page=import2&$TK" > import3.log
awk: line 1: syntax error at or near ,
* About to connect() to mydomain.com port 80 (#0)
* Trying 127.0.0.1... % Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0connected
* Connected to mydomain..com (127.0.0.1) port 80 (#0)
* failed creating formpost data
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0* Connection #0 to host mydomain.com left intact
curl: (26) failed creating formpost data
* Closing connection #0
Any ideas why the awk command is wrong on the server?
@lisa I had the same problem and seems to be caused by the cron job using a limited shell. Try placing this before the third curl line to use perl to extract the formtoken value
TOKEN=$(perl -ne 'print "$1" if /name="formtoken" value="(.{32})"/' import2.log)
echo $TOKEN
and then replace the awk command in the third curl command with
For anyone who is interested, I have found the solution for this…
Using Chrome, and using the Developer Tools, ie. under Customise > More tools > …or use ctrl & shift & I
Go through the step that you need to do to clear a list. Having done that
By right clicking on the ?page > Copy > Copy as cUrl (cmd) you can paste the command to notepad.
PS - The same approach can be used for any web page!
With this, and Duncan’s script details given earlier, we can derive the following…