Something bad happened / rsync

Q: Date/Time this occurred (Provide your time zone also)
A: 2022-09-07 06:01 AM AST

Q: DevKinsta Version
A: 2.7.0.4846

Q: OS Version
A: Mac OS 12.5 (M1)

Q: Docker Desktop Version
A: 4.11.1 (84025)

Q: Were any error codes or messages observed? If so, what were they?
A: Something bad happened

Q: Detailed Description of the Problem
**A: I am trying to sync a remote staging site with my local copy. It gets to about 1.6GB of 1.7 and then generates the error: Something bad happened. I’ve tried restarting several times. The download step of the sync process takes several minutes (>5 minutes). I checked the log, I saw that during that step, rsync is running. When I manually run rsync, it completes in a minute. I’m not sure what DevKinsta is doing for rsync that forces it to download every single file. See the log below.

[2022-09-07 06:01:25.473] [warn]  [terminalExec] Error: 
rsync: connection unexpectedly closed (8573955 bytes received so far) [receiver]

[2022-09-07 06:01:25.480] [warn]  [terminalExec] Error: rsync error: error in rsync protocol data stream (code 12) at io.c(228) [receiver=3.2.3]

[2022-09-07 06:01:25.483] [warn]  [terminalExec] Error: rsync: connection unexpectedly closed (1127478 bytes received so far) [generator]
rsync error: timeout in data send/receive (code 30) at io.c(228) [generator=3.2.3]

[2022-09-07 06:01:25.538] [info]  [ProgressIndicator] { isFailed: false, isOpen: true }
[2022-09-07 06:01:25.685] [info]  [terminalExec] Child process exited with code 30
[2022-09-07 06:01:25.688] [error] [downloadSite] Error while downloading files Error: Encountered an error in child process 50625: 30. rsync: connection unexpectedly closed (1127478 bytes received so far) [generator]
rsync error: timeout in data send/receive (code 30) at io.c(228) [generator=3.2.3]

    at ChildProcess.<anonymous> (/Applications/DevKinsta.app/Contents/Resources/app.asar/main.prod.js:2:821249)
    at ChildProcess.emit (node:events:394:28)
    at ChildProcess.emit (node:domain:475:12)
    at maybeClose (node:internal/child_process:1064:16)
    at Socket.<anonymous> (node:internal/child_process:450:11)
    at Socket.emit (node:events:394:28)
    at Socket.emit (node:domain:475:12)
    at Pipe.<anonymous> (node:net:672:12)
[2022-09-07 06:01:25.696] [error] [ipcMainStep] Error in operation SITE_PULL, step rsync: Error: Encountered an error in child process 50625: 30. rsync: connection unexpectedly closed (1127478 bytes received so far) [generator]
rsync error: timeout in data send/receive (code 30) at io.c(228) [generator=3.2.3]

    at ChildProcess.<anonymous> (/Applications/DevKinsta.app/Contents/Resources/app.asar/main.prod.js:2:821249)
    at ChildProcess.emit (node:events:394:28)
    at ChildProcess.emit (node:domain:475:12)
    at maybeClose (node:internal/child_process:1064:16)
    at Socket.<anonymous> (node:internal/child_process:450:11)
    at Socket.emit (node:events:394:28)
    at Socket.emit (node:domain:475:12)
    at Pipe.<anonymous> (node:net:672:12)
[2022-09-07 06:01:25.757] [info]  [editGlobalPhpIni] Getting content from  /www/kinsta/public/hidden/.user.ini
[2022-09-07 06:01:25.758] [info]  [dockerUtil/getContainer] Get 'devkinsta_fpm' Docker container
[2022-09-07 06:01:25.774] [info]  [ProgressIndicator] { isFailed: true, isOpen: true }
[2022-09-07 06:01:25.850] [info]  [containerExec] Command 'test -f /www/kinsta/public/hidden/.user.ini && echo $?' on devkinsta_fpm finished with exit code 1
[2022-09-07 06:01:25.851] [info]  [PhpIniEditor] Loading content into editor:

Hi @shomari, thanks for reaching out. Every time I see this sort of issue with rsync, it’s due to large files/directories on the origin site. It’s usually old backup/zip files or node_modules/.git directories. Moving those out of ~/public before the pull usually helps it complete.

Unfortunately there’s currently no way to exclude directories/files from the Pull, however we are working on that feature.

Please try deleting or moving any large and unnecessary files before trying again. I’m assuming the bulk of your site size is coming from the files and not the database.

Ok thanks. After a few attempts it worked but now I’m back at it again since I decided to start using GIT. It looks like the DevKinsta dashboard doesn’t include .git when calculating the files but it actually does attempt to transfer them. For now, I’ll move the .git folder before syncing.

1 Like

We’ve been having similar problems over here. We have a site with some very large files. The backup is nearly 2Gb in size. The automatic sync always fails during the download step. You can retry multiple times and it will get to different places but generally always fail. This has been tried now on 4 different systems 3 Linux Ubuntu 22.04 and a Mac running 10.5.1.

The solution has been to generate a downloadable backup of the desired environment in Kinsta, and install it manually with the recipe found here: Manually Import a Backup - Kinsta®.

I’m not familiar with where the log files are on the Mac platform (which is what I’m using now), but if you want additional data, point me to where those are and I can provide.

My take is that the download process in DevKinsta isn’t robust enough to recover from certain network glitches that cause it to fail. A retry starts over, which is not at all useful given the size of the site. Interestingly enough, the file download process built into the browser has no problems whatsoever with what I presume is the same backup file DevKinsta doesn’t seem to be able to handle.

My gut feeling is that if the backup download were to be improved, even if it just resumed where it left off, this problem would for all purposes go away.

I have the same problem with a site containing 10gb of media files. I have tried many times, but have failed to download the site to DevKinsta. Always get the “something bad happened” message.

DevKinsta needs a much more resilient method of downloading sites. This is a deal breaker.

Hi @patmccormick

Welcome to the Kinsta Community!

I am sorry to hear that you are facing such issues. To better assist you, can you please provide the following information?

DevKinsta Version:
OS Version:
Docker Desktop Version:

Please also send me the DevKinsta’s main.log file via private message. You can get the main.log file by clicking on the “Reveal log file in File Manager” under the Help and Support page in Devkinsta.

Thanks!

This is an irritating problem. When there is the vague error “something bad happened”, I can see that retrying first erases all the files already successfully downloaded, which makes no sense. Why doesn’t the process resume at the point it was? Why does it erase and then download the exact same files?

I also tried downloading a manual backup from Kinsta and creating a site from that. Although the file structure seems correct, the database is empty and the wp-config file is filled with hundreds of links and none of the standard content for that file. So, that approach does not function either.

I am on Mac Sequoia/Apple silicon.

Kindly check your Docker Desktop for these specific warning messages. If you see any, please click on them and then try the import again.


If the DevKinsta Pull action continues to fail and there is no existing working local site running, it would be best to just delete the existing Devkinsta Containers, Images, and Volume inside the Docker desktop and relaunch DevKinsta after to start fresh.

I hope the above recommendations help fix the issue, if not, please send us the Devkinsta main.log file. You can also manually locate the file typically at this directory “/Users//Library/Logs/DevKinsta”`

Let us know how it goes.

Here is my main.log:
main.log (869.3 KB)

It pulls the site completely but can’t launch the downloaded site due to a corrupt wp-config.php file. The log shows a number of errors.

I have alreadey tried reinstalling Docker, all Docker volumes & reinstalling Dev Kinsta. This problem persists.

I’m on Mac Sequoia with Apple silicon. latest versions of DevKinsta & Docker. Site size is about 10G.

Hello @patmccormick :wave:

I have checked the main.log file and the error shown there isn’t specific enough to pinpoint any part of wp-config.php code as the culprit. However, as you noticed yourself, the wp-config does seem to be the reason this is failing.

I would like to suggest another workaround if you haven’t tried it. Can you please try:

  • creating a new empty site on DevKinsta
  • if successfull, choose Sync → Pull from Kinsta and pull your site
  • You have the option not to sync all files, open the drop down and select not to sync wp-config
  • if the process fails on any step, try ‘Retry’ a couple of times

Once done, please let me know if that works.

Kind regards