Unable to sync from Live

DevKinsta Application Support
1

I am unable to sync a site.

[2023-11-06 12:09:31.810] [warn] [terminalExec] Error: [sender] io timeout after 60 seconds – exiting
rsync error: timeout in data send/receive (code 30) at io.c(204) [sender=3.1.3]

[2023-11-06 12:09:31.847] [warn] [terminalExec] Error:
rsync: connection unexpectedly closed (11489375 bytes received so far) [receiver]

[2023-11-06 12:09:31.851] [warn] [terminalExec] Error: rsync error: error in rsync protocol data stream (code 12) at io.c(231) [receiver=3.2.7]

[2023-11-06 12:09:31.919] [info] [ProgressIndicator] { isFailed: false, isOpen: true }
[2023-11-06 12:09:32.064] [warn] [terminalExec] Error: rsync: [generator] write error: Broken pipe (32)
rsync error: error in socket IO (code 10) at io.c(848) [generator=3.2.7]

[2023-11-06 12:09:33.865] [info] [terminalExec] Child process exited with code 10
[2023-11-06 12:09:33.873] [error] [downloadSite] Error while downloading files Error: Encountered an error in child process 70600: 10. rsync: [generator] write error: Broken pipe (32)
rsync error: error in socket IO (code 10) at io.c(848) [generator=3.2.7]

at handleChildProcessClosure (/Applications/DevKinsta.app/Contents/Resources/app.asar/main.prod.js:2:733315)
at ChildProcess.<anonymous> (/Applications/DevKinsta.app/Contents/Resources/app.asar/main.prod.js:2:732272)
at ChildProcess.emit (node:events:513:28)
at ChildProcess.emit (node:domain:489:12)
at maybeClose (node:internal/child_process:1091:16)
at Socket.<anonymous> (node:internal/child_process:449:11)
at Socket.emit (node:events:513:28)
at Socket.emit (node:domain:489:12)
at Pipe.<anonymous> (node:net:313:12)

[2023-11-06 12:09:33.882] [error] [ipcMainStep] Error in operation SITE_PULL, step rsync: Error: Encountered an error in child process 70600: 10. rsync: [generator] write error: Broken pipe (32)
rsync error: error in socket IO (code 10) at io.c(848) [generator=3.2.7]

at handleChildProcessClosure (/Applications/DevKinsta.app/Contents/Resources/app.asar/main.prod.js:2:733315)
at ChildProcess.<anonymous> (/Applications/DevKinsta.app/Contents/Resources/app.asar/main.prod.js:2:732272)
at ChildProcess.emit (node:events:513:28)
at ChildProcess.emit (node:domain:489:12)
at maybeClose (node:internal/child_process:1091:16)
at Socket.<anonymous> (node:internal/child_process:449:11)
at Socket.emit (node:events:513:28)
at Socket.emit (node:domain:489:12)
at Pipe.<anonymous> (node:net:313:12)

The log unfortunately does not tell what the command is that is being run, which would allow me to try to debug it. Amazingly the log reports [ProgressIndicator] { isFailed: false, isOpen: true } over and over again is which not very helpful.

Overall I have had terrible times trying to use the sync feature with DevKinsta. I am run a manual backup, but I do not think that is a few useful alternative.

Given that the backup of my local site takes a lot of time, it would be extremely useful if there were the ability to understand each step independently and to run them independently. This would support debugging and working towards a better solution.

2

Hi @Aporia

Thank you for reaching us out!

Right, the log does not tell what was the command that’s being run.
Anyway based on the logs you shared above, looks like it failed during the rsync process (and it timed out somehow).

Is this failing at the same percentage each time you try to sync from the Live site?
I’m wondering if is this due to a very large live site? It looks like the error is a general “timeout” error:

[terminalExec] Error: [sender] io timeout after 60 seconds – exiting
rsync error: timeout in data send/receive (code 30) at io.c(204) [sender=3.1.3]

Just a note that rsync can have issues with slower or unstable connections.
You might have to reduce the site’s size when necessary, like if you have single large files that you can manually copy over/delete that might help. Sometimes sites have leftover ZIPs or backups that don’t necessarily need to be transferred.

Best regards,
Agus

3

I keep seeing the size of the site matters, but since I do not know the command being run, I do not know what could be an issue.
Is it the database backup? Maybe it should be broken into separate sql files for each table to avoid this issue?
Or is it some other file?

Ultimately it would be great if the main.log provided more information about what is run. And it would also be great if DevKinsta allowed better interaction with each step of the sync, allowing us to skip steps if needed. This would help with debugging and also more atomic syncing for more complicated setups.

Also, I saw in the chat there is a plan to provide a way to skip files during the syncing. This would go a long way to helping with the above issue

4

Thank you for your reply :slight_smile:

Is it the database backup? Maybe it should be broken into separate sql files for each table to avoid this issue?
Or is it some other file?

As I could see (based on the log in the main.log file) the process when importing site (or when syncing site) from MyKinsta to DevKinsta is that the database will be dumped into .sql file (backup.sql) first and to be stored inside the site’s ~/public/ folder on the server side . No option to dump it into separate SQL files for each table.

Something like this for example (that I got in my own main.log when importing site or syncing site):

Command ‘ssh username@server.ssh.ip-address -p XYZXYZ -o “UserKnownHostsFile /dev/null” -o “StrictHostKeyChecking no” -o “PasswordAuthentication no” -o “ChallengeResponseAuthentication no” ‘mysqldump -h localhost -u dbusername -p****** databasename > “/www/username_xyz/public/backup.sql”’’ on devkinsta_fpm finished with exit code 0

after that DB dump process is completed/done, the site’s files inside the ~/public folder (including that backup.sql file) will be transferred (with rsync method) to our local computer.

And I agree that it’s difficult to see/understand what’s being run/transferred during that rsync process, as it would only just showing many of this line:

[info] [ProgressIndicator] { isFailed: false, isOpen: true }

Thank you for your feedback about the following too (these would be something really great and useful for us all indeed) :

Ultimately it would be great if the main.log provided more information about what is run. And it would also be great if DevKinsta allowed better interaction with each step of the sync, allowing us to skip steps if needed. This would help with debugging and also more atomic syncing for more complicated setups.

Also, I saw in the chat there is a plan to provide a way to skip files during the syncing. This would go a long way to helping with the above issue

I will pass what you mentioned/suggested above to our internal DevKinsta’s developer team, though I can’t promise anything yet if that’s something they can implement in the future (and can’t tell when that feature will be available in the future/next version of DevKinsta).

Best regards,
Agus

6

I do not have that ssh command in my main.log. Why would that be?

7

The ssh command to dump that database would be logged in the main.log (when we did the sync site) - I just tried once again while tailing the main.log file and grepped for the “/public/backup.sql” file as shown in the following screenshot:

image
image2009×670 148 KB

Depends on your operating system, if you’re on Mac or Linux, you may want to open Terminal program and go to the path/folder where the main.log file is saved (in my case on Linux is located inside the ~/.config/DevKinsta/logs ) and there, try to tailing the main.log file and to grepping the “/public/backup.sql” (similar to what’s showed in my screenshot above).

After that, try to open your DevKinsta and then try to: Sync → Pull from Kinsta , and (at the 3rd step “Creating DB dump” in DevKinsta) please watch the Terminal, if it will return that ssh command in question.

Best regards,
Agus

8

I think I must have cleared my main.log. I do have that backup created on my remote.
In fact, I was able to do a sync of just the database and it worked fine.

I know I can specify what directories to sync, however that does not make it easy for me to exclude things like .mov or other large files I have in the uploads file. Ultimately I am hoping to sync the database and get any new uploads, excluding large files

9

Yeah, if the “backup.sql” file is there on your remote folder (inside the ~/public folder) then that one most likely was created by that ssh /mysqldump command I mentioned before .
Glad to hear that you were able to do a sync of just the database which worked fine.

I understand that’s not easy to exclude them (things like .mov or other large files) as currently there’s no such option/feature in DevKinsta to exclude files based on file extension(s) nor based on file’s size
The current option is only to select or deselect any files and/or folders to be synced, something like this:

image
image604×773 34.1 KB

Best regards,
Agus

10

Before trying to use DevKinsta, I have developed various processes and scripts to sync a remote site and database. I will continue to use these, as they offer me more control. Although my whole goal in exploring DevKinsta is to enable less technical coworkers to use it.
I would seriously consider a better means of filtering and excluding files and file types, on an ongoing basis (ie. saved), so that my team can use DevKinsta.

11

Yep, if you’ve experienced in those various processes and scripts to sync a remote site and database, that would be very cool and awesome - and indeed those would offer you more control for sure!

Fully understand with your concerns, and I do agree with what you’ve mentioned above - that would be very helpful if there are such features in DevKinsta like filtering/excluding files and file types and whatnot.

I will pass this to our internal DevKinsta developers as well, so they can check and review this feature request (see if it can be implemented in the future/next version of DevKinsta), but again I can’t promise anything about it. Thank you once again for your feedback and suggestion!

Best regards,
Agus