From ckloiber at ckloiber.com Mon Oct 7 02:21:23 2002 From: ckloiber at ckloiber.com (Chris Kloiber) Date: 06 Oct 2002 21:21:23 -0400 Subject: Video Extraction (was: Re: [Extractstream [mailman]] Series 2 Tivo & USB Ethernet) In-Reply-To: <003501c26661$1b768c10$6701a8c0@silverbox> References: <003501c26661$1b768c10$6701a8c0@silverbox> Message-ID: <1033953683.28831.2.camel@athlon.ckloiber.com> I'm still dreaming of a reliable way to pull good quality video off of my series 1 with TiVoNet (my computer runs only Red Hat Linux 8.0) On Fri, 2002-09-27 at 16:04, Jay Haverty wrote: > I have a Series 2 Tivo and just used a USB Ethernet adapter to get it on > my network. I've been looking through the archives on Yahoo trying to > find a good solution for video extraction, but am having difficulties. > > Is there anyway to extract video from the Tivo 2 without having to open > it up? > > If not, what's my next best option for how to extract video? > > Thanks in advance, > > jay > > > _______________________________________________ > Extractstream mailing list > Extractstream@lists.merlins.org > http://lists.merlins.org/lists/listinfo/extractstream -- Chris Kloiber From Gifford.Hesketh at exodus.net Fri Oct 11 21:09:09 2002 From: Gifford.Hesketh at exodus.net (Gifford Hesketh) Date: Fri, 11 Oct 2002 13:09:09 -0700 Subject: [Extractstream [mailman]] streamserver Message-ID: <45258A4365C6B24A9832BFE224837D5582E124@SJDCEX01.int.exodus.net> where can I find it ? From weyer at apple.com Sun Oct 13 18:30:45 2002 From: weyer at apple.com (Thomas Weyer) Date: Sun, 13 Oct 2002 10:30:45 -0700 Subject: [Extractstream [mailman]] It must be user error.... In-Reply-To: <45258A4365C6B24A9832BFE224837D5582E124@SJDCEX01.int.exodus.net> References: <45258A4365C6B24A9832BFE224837D5582E124@SJDCEX01.int.exodus.net> Message-ID: ---------------------- multipart/alternative attachment I dl'd TiVoApp.zip from http://home.earthlink.net/~garyw90/TivoApp.html and thought I followed the instructions. I already had TiVoWeb running already 1.9.4b4(?). Ethernet and TiVoWeb working fine on my Sony w/ 2 120G hds. Now the problem. 1. I copied ShowList.tcl (ftp binary transfer from local computer here to TiVo) to the directory that has the main TiVoWeb.tcl file. I chmod'd it for execution (chmod +x ShowList.tcl)and confirmed it's setting (ls -l). Listed the file and looks ok. I tried chown'ing the file to match the others but get an Illegal Instruction. bash-2.02# ls -l total 174 -rw-rw-r-- 1 1000 1000 1381 Sep 22 06:59 Changelog -rw-rw-r-- 1 1000 1000 15127 Sep 22 06:59 GPL -rw-rw-r-- 1 1000 1000 353 Sep 22 06:59 Makefile -rw-rw-r-- 1 1000 1000 9117 Sep 22 06:59 README -rwxr-xr-x 1 0 1000 27360 Oct 12 07:21 ShowList.tcl -rw-rw-r-- 1 1000 1000 1961 Sep 22 06:59 TODO -rwxrwxr-x 1 1000 1000 23808 Sep 22 06:59 bsearch -rw-rw-r-- 1 1000 1000 3548 Sep 22 06:59 bsearch.c -rw-rw-r-- 1 1000 1000 1027 Sep 22 06:59 copyright -rw-rw-r-- 1 1000 1000 2146 Sep 22 06:59 daynight.css -rw-rw-r-- 1 1000 1000 11056 Sep 22 06:59 dtivogenre.js -rw-rw-r-- 1 1000 1000 7601 Sep 22 06:59 genre.js -rw-rw-r-- 1 1000 1000 8570 Sep 22 06:59 html.itcl -rw-rw-r-- 1 1000 1000 633 Sep 22 06:59 httpd-tt.itcl -rw-rw-r-- 1 1000 1000 21601 Sep 22 06:59 httpd-tt.tcl drwxrwsr-x 2 1000 1000 1024 Sep 22 06:59 images -rw-rw-r-- 1 1000 1000 10006 Sep 22 06:59 keytrap.js -rw-rw-r-- 1 1000 1000 6420 Sep 22 06:59 memrchr.c drwxrwsr-x 2 1000 1000 1024 Oct 12 08:11 modules -rw-rw-r-- 1 1000 1000 1284 Sep 22 06:59 sortof.css -rw-rw-r-- 1 1000 1000 1973 Sep 22 06:59 technophobe.css -rwxrwxr-x 1 1000 1000 653 Sep 22 06:59 tivoweb -rw-rw-r-- 1 1000 1000 70 Sep 22 06:59 tivoweb.cfg drwxrwsr-x 2 1000 1000 1024 Sep 22 06:59 uploads -rw-rw-r-- 1 1000 1000 4090 Sep 22 06:59 util.itcl bash-2.02# pwd /var/hack/tivoweb-tcl bash-2.02# chown 1000 ShowList.tcl Illegal Instruction 1a. For what it is worth ,the 2 other TiVoWeb modules I copied into the modules directory also don't work right (space.itcl, spaceused.itcl) though they do show up in TiVoWeb and try to launch. Could just be they aren't compatible with the current beta. This is less a concern then getting ShowList/ExtractStream to work but I though they might be related. Everything else in TiVoWeb that's part of the std distro works fine. bash-2.02# ls -l total 265 -rw-rw-r-- 1 1000 1000 17060 Sep 22 06:59 index.itcl -rw-rw-r-- 1 1000 1000 4978 Sep 22 06:59 info.itcl -rw-rw-r-- 1 1000 1000 4735 Sep 22 06:59 logos.itcl -rw-rw-r-- 1 1000 1000 1900 Sep 22 06:59 logs.itcl -rw-rw-r-- 1 1000 1000 7484 Sep 22 06:59 mail.itcl -rw-rw-r-- 1 1000 1000 3456 Sep 22 06:59 mfsbrowser.itcl -rw-rw-r-- 1 1000 1000 10317 Sep 22 06:59 phone.itcl -rw-rw-r-- 1 1000 1000 8687 Sep 22 06:59 resources.itcl -rw-rw-r-- 1 1000 1000 17288 Sep 22 06:59 sched.itcl -rw-rw-r-- 1 1000 1000 9043 Sep 22 06:59 screen.itcl -rw-rw-r-- 1 1000 1000 15856 Sep 22 06:59 search.itcl -rw-rw-r-- 1 0 1000 5900 Oct 12 04:58 space.itcl -rw-rw-r-- 1 0 1000 14371 Oct 12 05:00 spaceused.itcl -rw-rw-r-- 1 1000 1000 827 Sep 22 06:59 theme.itcl -rw-rw-r-- 1 1000 1000 2435 Sep 22 06:59 tvres-2.0.res -rw-rw-r-- 1 1000 1000 3440 Sep 22 06:59 tvres-2.5.res -rw-rw-r-- 1 1000 1000 3440 Sep 22 06:59 tvres-3.0.res -rw-rw-r-- 1 1000 1000 113959 Sep 22 06:59 ui.itcl -rw-rw-r-- 1 1000 1000 12134 Sep 22 06:59 webremote.itcl bash-2.02# pwd /var/hack/tivoweb-tcl/modules 2. I then copied ExtractStream and nc (ftp binary transfer from local computer here to TiVo) to my /var/hack directory. This directory is in my current path. I then I chmod'd them for execution (chmod +x ExtractStream nc) I then launched ExtractStream on the TiVo to test file execute. Got the cmd list for both and told myself all ways well. -rwxr-xr-x 1 0 0 60284 Oct 12 07:26 ExtractStream drwx------ 2 37330 4000 1024 Sep 22 06:47 bin -rwxrwxr-x 1 500 506 236076 Sep 22 06:41 joe -rwxr-xr-x 1 0 0 256000 Sep 22 00:28 joe.tar -rw-rw-rw- 1 500 506 17169 Sep 22 06:41 joerc -rwxr-xr-x 1 0 0 24736 Oct 12 07:26 nc -rwxr-xr-x 1 0 0 2693120 Sep 22 00:26 tbin.tar drwxrwsr-x 5 1000 1000 1024 Oct 12 07:21 tivoweb-tcl -rw-r--r-- 1 0 0 409600 Sep 22 06:55 tivoweb-tcl-1.9.4.beta3.tar bash-2.02# pwd /var/hack 3. I then launched TivoApp on a PC on my home net. It got the Tivo and displays TiVo web and the bash shell but gave an error connecting to/starting ShowList (couldn't find) I think I am all set but for some reason ShowList isn't working. One further question has anyone come up with an equivalent to TiVoApp for Mac OS X? I would rather NOT have to use the PC here for this as I have PLENTY of macs. Thanks for any help you can offer. -- Tom Weyer ---------------------- multipart/alternative attachment An HTML attachment was scrubbed... URL: http://lists.merlins.org/archives/extractstream/attachments/20021013/3b332817/attachment.html ---------------------- multipart/alternative attachment-- From extract at dormon.com Fri Oct 18 14:56:04 2002 From: extract at dormon.com (extract@dormon.com) Date: Fri, 18 Oct 2002 14:56:04 +0100 (BST) Subject: [Extractstream [mailman]] So near and yet so far In-Reply-To: Message-ID: I'm so close to getting this all working. Series 1 tivo with two 120GB drives and running OS 2.5.5 with tivo web 1.9.4 extractstream 0.2 works but gives random extraction, I think this is the big disk problem? I've tried CVS but it doesn't seem to work for me, is this fixed in 0.3? sendstream 0.2 seems to pull the correct tyStream (very very slowly, I'm dumping it via NFS) but when I use splitstream I end up with a massive badchunk.ty file, and only about 10secs of video of a half hour show. Has anyone got this working with large disks in a Linux only environment? Many many thanks for any help, or I'm going to have to start deleting stuff :( Cheers Lee From tivo at 30below.com Fri Oct 18 22:02:25 2002 From: tivo at 30below.com (TivoMerch) Date: Fri, 18 Oct 2002 17:02:25 -0400 Subject: [Extractstream [mailman]] So near and yet so far In-Reply-To: References: Message-ID: <5.1.0.14.2.20021018165647.023afa78@mail.30below.com> At 14:56 10/18/2002 +0100, you wrote: >sendstream 0.2 seems to pull the correct tyStream (very very slowly, I'm >dumping it via NFS) but when I use splitstream I end up with a massive >badchunk.ty file, and only about 10secs of video of a half hour show. Short answer: don't use NFS. Use netcat (nc). More info on how to use netcat is at my site: http://tivo.30below.com/zmerch Long answer: if you *have* to use nfs, there are command line switches that you can use to improve the data xfer speeds greatly (no switches usually nets 300kbyte/sec -- with switches usually gets you 750-850kbyte/sec... altho I usually get 950-1000kbyte/sec thru netcat)... Also, have you tried vsplit? It works very well on streams that splitstream seems to barf on... >Has anyone got this working with large disks in a Linux only >environment? My Netcat instructions will work fine in Linux, altho I do all my video work on Win2K. However, the TMPGEnc instructions & whatnot won't stretch nearly as well... :-( >Many many thanks for any help, or I'm going to have to start deleting >stuff :( Yea, that always does suck... :-/ Anyway, HTH! Roger "Merch" Merchberger From scott at alfter.us Tue Oct 22 00:25:14 2002 From: scott at alfter.us (Scott Alfter) Date: Mon, 21 Oct 2002 16:25:14 -0700 Subject: [Extractstream [mailman]] So near and yet so far In-Reply-To: References: Message-ID: <20021021232514.GB27095@alfter.us> On Fri, Oct 18, 2002 at 02:56:04PM +0100, extract@dormon.com wrote: > I'm so close to getting this all working. > > Series 1 tivo with two 120GB drives and running OS 2.5.5 with tivo web > 1.9.4 > > sendstream 0.2 seems to pull the correct tyStream (very very slowly, I'm > dumping it via NFS) but when I use splitstream I end up with a massive > badchunk.ty file, and only about 10secs of video of a half hour show. > > Has anyone got this working with large disks in a Linux only > environment? Have you tried using netcat instead of NFS? I use something like this on the TiVo side: sendstream -s xxxxxxx ... | nc -n -w 15 192.168.100.2 1200 ("xxxxxxx ..." is the set of tyStream numbers; 192.168.100.2 is the IP address of the receiving machine.) On the receiving machine (mine's Win2K, but Linux should be the same), use this: nc -l -p 1200 >foo.ty Start the receiver before the sender, then wait for the video to come through. Once you have the video, process it in the usual manner...don't know how splitstream works, but TyTool works fairly well. TyTool's Win32-only, though...if splitstream won't get the job done for you, maybe you can try running TyTool under Wine or VMware. _/_ / v \ Scott Alfter (salfter@salfter.dyndns.org) (IIGS( http://salfter.dyndns.org \_^_/ pkill -9 /bin/laden From tivo at 30below.com Wed Oct 30 06:14:29 2002 From: tivo at 30below.com (TivoMerch) Date: Wed, 30 Oct 2002 01:14:29 -0500 Subject: [Extractstream [MM]] Calling all Gurus!!! Tivo Technical questions... Message-ID: <5.1.0.14.2.20021030004110.00a6bb90@mail.30below.com> [[ Sorry for the x-post, but I did want this to reach as many Gurus as possible...]] The long awaited day has arrived! I have *finally* started work again on zss!!! I pared 15 pages of zss version 0.55 source, down to 5 pages of mostly /*comments*/, #declares, and *minor* code that I didn't want to reinvent (filehandling, etc.) and whacked, basically, the meat & potatoes (all the splitting code, which wasn't compatible with UK streams, all the audio-offset code, which didn't always work... etc...) and I'm starting, basically, from scratch. I have *already* learned a bit more about the Tivo's header records... and my program will have the ability of logging all of this if you wish... and I have some questions as to what I actually need to pay attention to, and other things I *think* might be odd (as I've never actually looked at this stuff - I only made logging / usability adjustments to Joe's code before) but I'm not sure, so I could use a little advice: 1) Am I Nutz??? [[ well, I already have the answer: Hell Yes!!! ;-) ]] 2) PES Header data - I've got it (but I don't have it decoded correctly from hex, yet... minor bug...) and I've looked at it, and it confuses me... Do I even need to pay any attention to it??? 3) Many "tivo" records have the same PES records but different data sizes -- at first I thought I b0rked something, so at the end of each chunk, I told the proggie to add up all the audio & video record sizes, throw in the Tivo Header size, and spit out a total -- most of the time, it was 0x20000 - so to me, it says that I'm not totally screwed up; but on occasion it was from 0x1FFE8 to 0x1FFFC, on multiples of 4 bytes. I would think that this would be too small if it were actually ClosedCaption data, [[ which types 0x01 & 0x02 are "supposed" to be, but quite often, they had a size that was knackered up, and the rest of the record either seemed to have a normal PES, or was 0's... -- I guess my question is this - Are Tivo records linear to the actual audio/video records in the MPEG stream, or do I not care about actual sizes, other than my program playing "traffic cop" - all C0 & D0 packets going to the .m2a, and all the E0 & F0 packets going to the .m2v, and bitbucket the rest??? Well, instead of asking a perjillion questions, I'll close it with: "And any other Tivo header info you might have on hand -- I'd love! (For me & my proggie, but also for my Tivo header page, so others can benefit, too!!! ;-) Anywho, Thanks tons, and laterz! :-) Roger "Merch" Merchberger From faye at clara.net Wed Oct 30 10:47:15 2002 From: faye at clara.net (Faye Pearson) Date: Wed, 30 Oct 2002 10:47:15 +0000 Subject: [Extractstream [MM]] Calling all Gurus!!! Tivo Technical questions... In-Reply-To: <5.1.0.14.2.20021030004110.00a6bb90@mail.30below.com> References: <5.1.0.14.2.20021030004110.00a6bb90@mail.30below.com> Message-ID: <20021030104715.GB3901@clara.net> TivoMerch [tivo@30below.com] wrote: > I have *already* learned a bit more about the Tivo's header records... and > my program will have the ability of logging all of this if you wish... and > I have some questions as to what I actually need to pay attention to, and > other things I *think* might be odd (as I've never actually looked at this > stuff - I only made logging / usability adjustments to Joe's code before) > but I'm not sure, so I could use a little advice: Check out my thread on http://alt.org/forum/ (Towards getting a complete explanation of MFS TyStream chunk directory) http://alt.org/forum/index.php?t=msg&th=15&start=0&rid=&S=afed37df20b5dac275b61274319e4bf7 I did a lot of research, but haven't had much time to complete my own program. Faye -- Faye Pearson, Covert Development ClaraNET Ltd. Tel 020 7903 3000 Remember, there's a big difference between kneeling down and bending over. - Frank Zappa From tivo at 30below.com Thu Oct 31 04:02:14 2002 From: tivo at 30below.com (TivoMerch) Date: Wed, 30 Oct 2002 23:02:14 -0500 Subject: [Extractstream [MM]] Calling all Gurus!!! Tivo Technical questions... In-Reply-To: <200210300721.g9U7LOu47797@minnie.tuhs.org> References: <5.1.0.14.2.20021030004110.00a6bb90@mail.30below.com> Message-ID: <5.1.0.14.2.20021030222747.00ab9e58@mail.30below.com> Of the two guru replies I got, both came from this list, so I figgered there's no need to send it thru the Yahoo list, too, so... At 17:21 10/30/2002 +1000, Warren wrote: >Ok, I suppose I should 'fess up. I've been working on Playitsam version 2.... [snip] >What do you think? > >Cheers, > Warren Well, with a little begging, pleading, and a promise that I wouldn't publish the code as it's unfinished, Warren sent me the the source to V.2... and without letting the cat out of the bag, lemme just say this is *way* deeper than I'd have been able to create on my own [at least not for a *very* long time...]. My color laser printer happily churned out almost 30 pages of notes, code & whatnot, and that's just to get started in what should be a very fun, but intense, learning exercise. Warren - My hat's off to you! I don't plan on 'borrowing' any code - from first look, what I'm trying to do is quite a bit different than playitsam, but if I do need a small clip, I'll make sure you get credit! However, you have a few ideas that are somewhat similar to what I wanted to do in zss, so I'm sure there's a lot of concepts that I'll be able to glean from your code & learn from... I've been programming for a long time, in languages as strange as APL & Lisp [most of my current expertise being in Perl] but I'm still a relative beginner in C, so I still have a lot of catch-up to do there. If there's anything I can do to help [I'm not bad at adding bells & whistles to things - I did make quite a few changes to splitstream folks liked, so if there's anything I can do to help, just holler! ;-) and... At 10:47 10/30/2002 +0000, Faye wrote: >Check out my thread on http://alt.org/forum/ (Towards getting a complete >explanation of MFS TyStream chunk directory) > >http://alt.org/forum/index.php?t=msg&th=15&start=0&rid=&S=afed37df20b5dac275b61274319e4bf7 I did... and the good news, is it's excellent stuff! :-)) The only bad news is, the forum doesn't have a "printer-friendly" button, and a lot of the text & data gets "clipped" off the edge printing in portrait in both IE & Mozilla 1.0.1... :-( I'll try "selecting" the text next, and/or landscape mode to see if I can get a decent copy onto paper... (despite being a tree-hugger, sometimes the only way to get big ideas into my widdow pea brain is by putting it on thin, flat, white bark... ;-) If that doesn't work, I'll figure out how to get that rascal into dreamweaver - it can reformat darned near everything! ;-) >I did a lot of research, but haven't had much time to complete my own >program. And wow, did you! Excellent job! \/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/ And now, a few notes & whatnot that I'd figured out on my own: I like logging. *Lots* of logging. Logs get big, pkzip makes 'em small again! ;^> In my initial rework of zss (ver. 0.60 at this point...) just the tivo header info on 4480 chunks created over 41Megs of logs, to help try to figure things out for a good, accurate split, and this is what I found: [[ people smarter than me might know this, but hey... gotta put something in the archives! ;-) ]] From what I can see, the PES data in the Tivo Chunk headers means very little to the actual MPEG stream itself - I think it's for internal Tivo housekeeping... Also, it looks like the chunk boundaries mean very little to the actual audio / video boundaries so care needs to be taken to make sure that streams aren't mangled in mid-chunk... I've already learned a lot more in the last few days on how the tivo headers look & work, so I'll be updating my tivo header page shortly... remember, if anyone has any info that would help complete the info on there, feel free to email it my way! ;-) Many Thanks, all! I'm running a little late for Law & Order... but... I have a Tivo! ;-) Laterz, Roger "Merch" Merchberger