Help on uploading dropouts and loss.
 

After 37 years as a producer and director I am semi retired and doing little projects that have me editing etc. I am not a techie so I am learning as I go.

I have to upload the finished edits to servers. I have had massive problems getting my finished edits to the hosting servers. This will be long but I need to cover all the bases.

I have Comcast cable internet service. I started with the basic speed. I was using a version of Ipswitch imbedded in AD/Dsee a photo/ video management program. It worked pretty well but the uploads were slow.

I upgraded to a higher level of Comcast service. They call it 16 meg service. I get tested speeds of about 3,000KBs up and Downloads at 15,900 to 21,600 KBs. Using Speakeasy testing.

With the new speed came problems with interrupted and lost in the middle uploads.

I purchsed a new copy of Ipswitch and installed it.

The servers I am uploading to are with companies that do business because they have massive capabilities to handle lots of incoming uploads so I know the problem isn't on their end. One of them also had their own FTP program available to their customers. I have used it too.

I have had a tech look into it a little bit, but with almost no success. I have spent time talking to Comcast. Three weeks ago I had a Comcast service guy here and one thing he told me was that the signal inside the house is lower than when the cable arrives at the house. I am in an area that the cable runs between the property(ies) is long. We are on 2 acre or more lots. The run from the box to the house is at least 150 feet and there is at least another 140 feet inside before the cable gets to the computers. The COmcast tech added an inline amplifier inside the house. The cable run from the box to the house was upgraded two years ago to a better cable. The run into the house dates to 1991.

Here is where I am on this. Up until Friday I was lucky to get 1 out of 10 attempts to upload properly at the new speed. Saturday I tried the laptop outside at the point the new cable run makes the house. No improvement there. I have been using the windows performance meters to track computer activity during the uploads. My tech tells me and the meters confirm that uploading is very memory dependent. The two computers I have been using for uploading each have 1 GB of memory. One is a Dell 350 work station and the other is a Dell M60 Workstation laptop.

Since I have to baby sit the uploads I sit at another computer and surf. I was on You Tube chasing around guitar players and low and behold the upload went the first time out. So I tried another upload while downloading music and it worked. Using this method has me to the point where I am 60 to 70% of the uploads successful the first time.

My tech seems to think that keeping the line busy has something to do with the success. Watching the memory trace on the uploading computer during the upload the trace can be everything from 20 to 95 with an average of about 65. Sometimes the trace is cluster to the top. Sometimes to the bottom half and sometimes it covers top to bottom. And, occassionally it will flat line down the center at about 60.

The best combination of machine, software and drive has been the M60 Laptop with a memory sitck and the uploading servers FTP program.

Sorry for being so long but I wanted to cover everything that got me to this point.

Any, help or thoughts would be appreciated.

First, 3000KBs is only 3MB or megs per second. Which means that Comcast owes you 13 MB of speed - their problem.

Two, get more memory for you machine 2 MB would be nice, 4 would be better.

Third, the Comcast cable is the video pipeline into and out of the house. If you are watching cable TV, surfing on another computer, using Skype, etc. you are using up valuable space in your pipeline. Your upload needs all the bandwidth you can provide and you are jamming it with your other activities.

Fourth, how many cable splitters are in the house? Each one drains signal and creates line loss and adversely affects your speed.

Finally, upload in the middle of the night when servers and the cable line (which you share with others) are most free of digital traffic.

Thanks for the suggestions.

In retrospect much of the problem I am having is after I switched to the faster service. So maybe going to the standard comcast service is the way to go.

I am in the process of getting a newer computer with 4 Gig of memory.

I have a router upstairs. I currently have two lines active. It is an 8 port router and the lines are plugged into it but they are not connected at the other end. Should I unplug the unused lines?

I have one splitter in the office and three computers are on the line.

Just added

I have a second line available for the office. I could dedicate this to the upload computer. The line would go from the upload computer to the router. Would that help?

Further update

I have the upload computer (Dell M60 Workstation laptop) on the straight line (Line #2) to the router. I am using a flash memory stick with the video transfered to it. The laptop has 1 Gig of memory.

I unplugged the other two computers from the first line (Line #1) that has a splitter in the office. So there is no other computer on line anywhere in the house.

No TV running either.

I am batting about 60% now which is better than it was.

Michael

One of my clients is very big in the internet business. They upload hundreds of hours of video a month and they have sent me information about editing PHP.ini files to change some settings. I will leave that for my tech when he gets here.

However, I really am beginning to think that the problems are beyond the computers here. I have been trying with two different Dell Workstations. A 3 year old Dell 350 with 1 Gig of memory and a Dell M60 Laptop workstation with 1 gig of memory.

Today I tried using the oldest of the Dell workstations I have. It is a 1998 420 Workstation. It is an 800 MHz processor and it has 256 MB of memory.
It is a boat anchor, but it runs flawlessly. I had the operating system upgraded to Windos 2000 a couple of years ago.

I tried uploading with this computer using a flash memory stick as the source. It uploads slower, but eventually I had the same problem. I got a 326 MB file (with indicated 20 minutes to upload) down to 1.5 minutes left before it failed. So close.

I really am beginning to think the problem lies in the cable, or at Comcast.

I have about run out of possible fixes.

Michael, When are you uploading? Remember that with Comcast cable internet you are "sharing" the cable connection with everyone else that is online from you to the switching station (I think that's what it's called). the more people that are online the more bandwidth is taken up and the slower your upload will be - until it times out. Try to do your upload during slow periods such as late at night or mid day.

Slower upload speeds compared to download speeds are the norm for residential cable/DSL broadband. You're only likely to get equal upload/download speeds with a pricier business package.

Comcast stipulates for their residential cable Internet service that "Maximum upload speeds range between 384 Kbps and 2 Mbps depending on the service tier selected and can be even faster with PowerBoost®."

The 3 Mbps (megabits per second) Michael is getting is very fast for an upload. I'm in a state capitol and the fastest cable broadband upload I've seen available in my area is 1 Mbps with a 6 Mbps download package.

The amount of system RAM shouldn't make a difference with uploads. I uploaded several 1 GB (non-video) files in the last week to my Web space from a Windows XP computer with 1 GB of RAM using FileZilla. All went perfectly.

Reading about how cable television and Internet works, I highly doubt he's anywhere near maxing out the coaxial cable bandwidth, especially on upload. He should be able to watch TV and surf the Internet while uploading without issues.

Since Michael getting so many dropouts after upgrading his package when he didn't have any with the slower package, my guess is Comcast has a technical issue somewhere in their infrastructure that's affecting the stability of his faster service. He should not be having these issues with uploads. My advice to Michael: Keep after Comcast. They should be the ones to fix this. Or, if you don't want to hassle with it anymore and you can live with slower uploads, switch back to the lower speed package. Either way, you shouldn't be paying for a faster service that has so many issues.

His uploads didn't time out when he was on the slower tier with less bandwidth, only when he switched to the faster one. That seems to indicate to me that there's an issue involved other than available bandwidth. If it is bandwidth involved, then Comcast is either massively overloading the available capacity in his area and/or doing a very poor job of load balancing. Uploads should not grow so choked as to completely timeout. But if that is the case, switching back to the slower tier should not solve his issues.

The next two things we are going to try is (1) wireless broadband. My tech has a card.

(2) I am also trying to contact a professional photographer friend of mine to find out if he has a computer with more memory. I know he has cable. He is about 10 miles away frem here and the service may be better.

I appreciate all the help on this.

I'll add another update.

I have been in contact with one of the companies that I am uploading to and they say some of the problems with the large files (300MB and up); is coming from Comcast and Road Runner. I am not their only supplier having problems as of late.

They say, Comcast (et.al.) is under pressure from the motion picture industry and the music business to control file sharing. I don't know how much truth there is to this but I am going to research it tomorrow. I read one story about this the other night.

My wife is on the village board here and through the villiage local Comcast officials can be contacted. I am finding customer service to be worthless on this issue.

If I find out more I will post it at that time.

Ran some searches on this. From an Electronic Frontier Foundation report on Comcast's packet forgery:

"EFF has also received unconfirmed reports that Comcast is interfering with other protocols. In particular, some Comcast users have reported that medium and large-sized FTP and HTTP transfers have been interrupted. The FTP and HTTP reports, however, have not included enough detail for us to be certain that there is a problem attributable to forgery of packets by Comcast. Our attempts to test for interference in large HTTP transfers have occasionally resulted in what seem to be interrupted connections, but these results are not consistently reproducible, and we cannot say at this point that there is any interference or that it is caused by Comcast."

Full report (PDF): Packet Forgery By ISPs: A Report On The Comcast Affair

An interesting DSL Reports thread: Comcast is throttling FTP uploads

This might be worth looking into: FTP problems with Windows Server FTP service

And there's this: In about-face, Comcast to stop hampering of Internet file-sharing

...but don't get too excited. Comcast only says they will do so by the end of the year, so if Comcast is interfering with FTP traffic in addition to P2P you could still have issues for a while yet.

My only other suggestions at this point would be to try SFTP (Secure FTP), or preferably establish a VPN (Virtual Private Network) connection before uploading via FTP. If your upload goes smoothly through a VPN connection, then it may indicate Comcast is doing something that is messing with FTP uploads, whether purposely or as a side-effect of P2P traffic interference.

The finl update... I hope!
 

I did some internet snooping and it is pretty evident that Comcast has been causing problems with large files.

Here is one of the reports I found.

(03-27) 12:43 PDT SAN FRANCISCO -- Comcast Corp. promised Thursday that it would not discriminate against specific Internet traffic on its network, ending its controversial practice of targeting and blocking file-sharing technologies from San Francisco's BitTorrent Inc. and others.

The nation's largest cable TV and Internet provider said it would work with BitTorrent and other companies offering peer-to-peer file-sharing services to come up with improved ways to speed large files efficiently. Comcast said it would still manage traffic during peak periods but by year's end would not block specific applications - a recent practice that prompted federal scrutiny and pressure.

"This means that we will have to rapidly reconfigure our network management systems, but the outcome will be a traffic management technique that is more appropriate for today's emerging Internet trends," Tony Werner, Comcast's chief technology officer, said in a statement.............etc

Late today I spoke with a rep at Comcast and he confirmed the practice is taking place. The issue will be resolved by the end of the year. It is a combination of the file sharing issue and the huge demands already on the system. I am sure the growth of the internet is exponential and trying to stay ahead of the game is tough. Some form of electronics is involved in this monitoring and choking off of traffic and the hardware will have to be replaced was what was offered to me.

I spoke to the Comcast Business Division and they could not guarantee that they could do any better. It is all coming down to managing traffic in a system that is strained to its limits.

So, I have downgraded to basic internet with powerboost and I will attemt to upload full edits. If it does not go I will have to send it in sections to one of the clients. In the case of the other one I will send them DVD's and they will load it in their system and I can manage it from here.

Again, thanks to all who offered suggestions. Either I or my tech tried many of them. It just worked out that we could do nothing. The problem was bigger than both of us.