Import csv with cronjob

ℹ️ Support
1

Hello people,

im trying to import CSV files, but it stops all the time. Any chance to attach a cron job to the import function also?

2

Hi, I’ve had this problem too, the cause was that there were illegal characters in the email address. The symptom was that the import would get to a certain record (say the 950th record), and then go no further). When I looked at that record in the csv file, it had some weird characters in the email address.

When I deleted that record, and re-imported, it fixed the problem.

1 Like
3

I am trying to import emails in csv format but it is halting and I checked and no illegal characters I can spot. Please help

4

I prefer Tab Separated Values for import files.
This file removes email addys that have illegal chars.

  1. Copy the code below and save the file as “cleanaddys.php”.
  2. Create a file with one email addy per line and save it as “in.txt”.
  3. Upload both to your server.
  4. Visit the page (http://mysite.com/cleanaddys.php).
  5. Download the file “out.txt”.
  6. Delete the files from your server.

out.txt is a much better import file.
I know it’s crudely written, but I threw it together in less than 10 minutes so cut me some slack.
Enjoy

<?php

//    REMOVES BAD EMAIL ADDY'S... CREATES IMPORT FILE: OUT.TXT

if (file_exists("out.txt")) {unlink("out.txt");};
$cntinvalid=0;
$file = fopen("in.txt","r"); // Input file.
$file2 = fopen("out.txt","a"); // Output file.
    fwrite($file2, "Email\tSend this user HTML emails\tRSS Frequency\tIs this account disabled?\n");
    while(!feof($file)) {
        $myline=strtolower(fgets($file));
        $myline=rtrim($myline);
        $myline=rtrim($myline,";");
        if ((substr_count("$myline","@") <> 1) || ($myline[0] == '-')) {
            echo "$myline --- @ ERROR - NOT VALID!<br />";
            $cntinvalid++;
        } else {
            if((strpos("$myline",'`') == false) &&
                (strpos("$myline",'~' ) == false) &&
                (strpos("$myline",'!') == false) &&
                (strpos("$myline",'#') == false) &&
                (strpos("$myline",'$') == false) &&
                (strpos("$myline",'%') == false) &&
                (strpos("$myline",'^') == false) &&
                (strpos("$myline",'&') == false) &&
                (strpos("$myline",'*') == false) &&
                (strpos("$myline",'(') == false) &&
                (strpos("$myline",')') == false) &&
                (strpos("$myline",'+') == false) &&
                (strpos("$myline",'=') == false) &&
                (strpos("$myline",'[') == false) &&
                (strpos("$myline",'\{') == false) &&
                (strpos("$myline",'\}') == false) &&
                (strpos("$myline",']') == false) &&
                (strpos("$myline",'|') == false) &&
                (strpos("$myline",'\\') == false) &&
                (strpos("$myline",':') == false) &&
                (strpos("$myline",';') == false) &&
                (strpos("$myline",'"') == false) &&
                (strpos("$myline",'\'') == false) &&
                (strpos("$myline",'<') == false) &&
                (strpos("$myline",',') == false) &&
                (strpos("$myline",'>') == false) &&
                (strpos("$myline",'..') == false) &&
                (strpos("$myline",'?') == false) &&
                (strpos("$myline",'/') == false) &&
                (strpos("$myline",'\t') == false) &&
                (strpos("$myline",' ') == false)
                ) {

                fwrite($file2, "$myline\t1\t0\t0\n");
            } else {
                echo "$myline --- NOT VALID!<br />";
                $cntinvalid++;
            };
        };
    };
fclose($file);
fclose($file2);
$fileoutcnt = "out.txt";
$lineoutcnt = (count(file($fileoutcnt))-1);
echo "$cntinvalid addresses were invalid<br />$lineoutcnt are ready for import";

?>
5

Can you see at which line the import hangs?

6

How can I find out? do you have a method?

7

Hi,

Thank you so much but I have attributes need to be added to each email as well. what format do I need to save in.txt file (MS-DOS) Text or SCV Coma-delimited)?

8

plain text (MS-DOS or Unix)
One email addy per line.

My script creates just 3 attributes and presets them.
Send this user HTML emails = 1
RSS Frequency = 0
Is this account disabled? = 0

1 Like
9

It says:

Importing 35255 subscribers to 4 lists, please wait
Please wait
1970 / 35255

10

Thanks but it still halts on Importing 35255 subscribers to 4 lists, please wait
Please wait
1970 / 35255

11

check line 1969, 70 and 71 - any errors in the address? Bad characters reek havoc :smile:

12

Nop

(image removed by admin)

13

I would try and import just the first 1960.
Then try and import the next 50 and see if there’s a problem.
If there’s no problem, I would assume that the import script is timing out.

14

How can I increase that?

15

Server Management:
Settings within PHP Configuration Editor

It should not require a recompile.