Closed Bug 558496 Opened 15 years ago Closed 15 years ago

macosx64-debug - intermittent timeout loading bloatcycle.html (leaktest)

Categories

(Release Engineering :: General, defect, P3)

Product:
Release Engineering
Component:
General
Platform:
x86_64
macOS
Type:
defect

Tracking

(Not tracked)

Status:
RESOLVED FIXED

People

(Reporter: philor, Assigned: catlee)

References

Details

(Keywords: intermittent-failure, Whiteboard: [10.6])

Attachments

(1 file)

Disable tracemalloc tests on osx64
10.02 KB, patch
bhearsum
: review+
catlee
: checked-in+
Details | Diff | Splinter Review
http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1270846553.1270853473.16985.gz OS X 10.6.2 mozilla-central leak test build on 2010/04/09 13:55:53 s: moz2-darwin10-slave41 and then http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1270849536.1270856130.26144.gz OS X 10.6.2 mozilla-central leak test build on 2010/04/09 14:45:36 s: moz2-darwin10-slave47 both with the same pair of non-zero exit codes: ======== BuildStep started ======== checking clobber times failed === Output === python mozilla-central-macosx64-debug/tools/clobberer/clobberer.py -s tools -t 168 http://build.mozilla.org/clobberer/index.php mozilla-central OS X 10.6.2 mozilla-central leak test build mozilla-central-macosx64-debug moz2-darwin10-slave47 http://production-master.build.mozilla.org:8010/ in dir /builds/slave/mozilla-central-macosx64-debug/.. (timeout 3600 secs) ... Checking clobber URL: http://build.mozilla.org/clobberer/index.php?master=http%3A%2F%2Fproduction-master.build.mozilla.org%3A8010%2F&slave=moz2-darwin10-slave47&builddir=mozilla-central-macosx64-debug&branch=mozilla-central&buildername=OS+X+10.6.2+mozilla-central+leak+test+build Error contacting server program finished with exit code 1 elapsedTime=1.604003 === Output ended === ======== BuildStep ended ======== then ======== BuildStep started ======== alive test failed === Output === python leaktest.py -- --trace-malloc malloc.log --shutdown-leaks=sdleak.log in dir /builds/slave/mozilla-central-macosx64-debug/build/obj-firefox/_leaktest (timeout 3600 secs) ... localhost - - [09/Apr/2010 15:35:23] "GET /bloatcycle.html HTTP/1.1" 200 - command timed out: 3600 seconds without output, killing pid 27611 process killed by signal 9 program finished with exit code -1 elapsedTime=4129.535885 === Output ended === ======== BuildStep ended ========
As I sort of suspected, the clobber time check was transitory, and not just these, but the bloatcycle timeout continues. http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1270916096.1270921065.2475.gz OS X 10.6.2 mozilla-central leak test build on 2010/04/10 09:14:56 s: moz2-darwin10-slave17
Summary: macosx64-debug - intermittent "checking clobber times failed" and then timeout loading bloatcycle.html → macosx64-debug - intermittent timeout loading bloatcycle.html
http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1270923455.1270930265.20277.gz OS X 10.6.2 mozilla-central leak test build on 2010/04/10 11:17:35 s: moz2-darwin10-slave17
http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1270945213.1270952552.4242.gz OS X 10.6.2 mozilla-central leak test build on 2010/04/10 17:20:13
http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1270950129.1270954635.8161.gz OS X 10.6.2 mozilla-central leak test build on 2010/04/10 18:42:09
comment 3: s: moz2-darwin10-slave17 comment 4: s: moz2-darwin10-slave41 http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1270958369.1270965706.5382.gz OS X 10.6.2 mozilla-central leak test build on 2010/04/10 20:59:29 s: moz2-darwin10-slave17
http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1271024531.1271030592.13769.gz OS X 10.6.2 mozilla-central leak test build on 2010/04/11 15:22:11 s: moz2-darwin10-slave16
http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1271026113.1271033447.20813.gz Build Log (Full) OS X 10.6.2 mozilla-central leak test build on 2010/04/11 15:48:33 s: moz2-darwin10-slave47
http://tinderbox.mozilla.org/Firefox/1271039605.1271044953.14941.gz OS X 10.6.2 mozilla-central leak test build on 2010/04/11 19:33:25 s: moz2-darwin10-slave34
http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1271060468.1271065654.14864.gz OS X 10.6.2 mozilla-central leak test build on 2010/04/12 01:21:08 s: moz2-darwin10-slave34 http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1271058270.1271062805.3630.gz OS X 10.6.2 mozilla-central leak test build on 2010/04/12 00:44:30 s: moz2-darwin10-slave06
http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1271097636.1271103249.25765.gz OS X 10.6.2 mozilla-central leak test build on 2010/04/12 11:40:36 s: moz2-darwin10-slave05 http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1271087894.1271096264.7363.gz OS X 10.6.2 mozilla-central leak test build on 2010/04/12 08:58:14 s: moz2-darwin10-slave05
http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1271141491.1271146240.15771.gz OS X 10.6.2 mozilla-central leak test build on 2010/04/12 23:51:31 s: moz2-darwin10-slave06 http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1271149290.1271156041.14223.gz OS X 10.6.2 mozilla-central leak test build on 2010/04/13 02:01:30 s: moz2-darwin10-slave34
http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1271176538.1271181499.9437.gz OS X 10.6.2 mozilla-central leak test build on 2010/04/13 09:35:38 s: moz2-darwin10-slave34
http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1271181554.1271186748.26056.gz OS X 10.6.2 mozilla-central leak test build on 2010/04/13 10:59:14 s: moz2-darwin10-slave34
http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1271244783.1271249521.1417.gz OS X 10.6.2 mozilla-central leak test build on 2010/04/14 04:33:03
http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1271321543.1271326449.30973.gz OS X 10.6.2 mozilla-central leak test build on 2010/04/15 01:52:23 s: moz2-darwin10-slave27 http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1271323033.1271328218.5504.gz OS X 10.6.2 mozilla-central leak test build on 2010/04/15 02:17:13 s: moz2-darwin10-slave05
http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1271339577.1271344537.32343.gz OS X 10.6.2 mozilla-central leak test build on 2010/04/15 06:52:57 s: moz2-darwin10-slave27
http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1271324947.1271333029.23530.gz OS X 10.6.2 mozilla-central leak test build on 2010/04/15 02:49:07 s: moz2-darwin10-slave17
http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1271455069.1271462419.29731.gz OS X 10.6.2 mozilla-central leak test build on 2010/04/16 14:57:49 s: moz2-darwin10-slave27
http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1271780571.1271786321.30022.gz OS X 10.6.2 mozilla-central leak test build on 2010/04/20 09:22:51 s: moz2-darwin10-slave18
http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1271795284.1271804065.19673.gz OS X 10.6.2 mozilla-central leak test build on 2010/04/20 13:28:04 s: moz2-darwin10-slave18
http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1271800092.1271808547.31726.gz OS X 10.6.2 mozilla-central leak test build on 2010/04/20 14:48:12 s: moz2-darwin10-slave10
http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1271831113.1271837287.4095.gz OS X 10.6.2 mozilla-central leak test build on 2010/04/20 23:25:13 s: moz2-darwin10-slave17
http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1271839246.1271845682.443.gz OS X 10.6.2 mozilla-central leak test build on 2010/04/21 01:40:46 s: moz2-darwin10-slave17
http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1271841131.1271847581.6760.gz OS X 10.6.2 mozilla-central leak test build on 2010/04/21 02:12:11 s: moz2-darwin10-slave18
http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1272055780.1272061023.10053.gz OS X 10.6.2 mozilla-central leak test build on 2010/04/23 13:49:40 s: moz2-darwin10-slave22
http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1272060597.1272068941.1747.gz OS X 10.6.2 mozilla-central leak test build on 2010/04/23 15:09:57 s: moz2-darwin10-slave18
http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1272142467.1272147271.4043.gz OS X 10.6.2 mozilla-central leak test build on 2010/04/24 13:54:27 s: moz2-darwin10-slave18
http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1272145981.1272157763.23168.gz OS X 10.6.2 mozilla-central leak test build on 2010/04/24 14:53:01 s: moz2-darwin10-slave10
http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1272210054.1272214858.26715.gz OS X 10.6.2 mozilla-central leak test build on 2010/04/25 08:40:54 s: moz2-darwin10-slave10
http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1272340693.1272345876.18761.gz OS X 10.6.2 mozilla-central leak test build on 2010/04/26 20:58:13 s: moz2-darwin10-slave10
http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1272369002.1272376174.8468.gz OS X 10.6.2 mozilla-central leak test build on 2010/04/27 04:50:02 s: moz2-darwin10-slave05
http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1272370034.1272377721.14077.gz OS X 10.6.2 mozilla-central leak test build on 2010/04/27 05:07:14 s: moz2-darwin10-slave09
http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1272361281.1272367511.28148.gz OS X 10.6.2 mozilla-central leak test build on 2010/04/27 02:41:21 s: moz2-darwin10-slave18
http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1272358380.1272364271.14657.gz OS X 10.6.2 mozilla-central leak test build on 2010/04/27 01:53:00 s: moz2-darwin10-slave05
http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1272387652.1272395324.7787.gz OS X 10.6.2 mozilla-central leak test build on 2010/04/27 10:00:52 s: moz2-darwin10-slave18
http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1272394451.1272399435.20003.gz OS X 10.6.2 mozilla-central leak test build on 2010/04/27 11:54:11 s: moz2-darwin10-slave05
http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1272439290.1272445104.17478.gz OS X 10.6.2 mozilla-central leak test build on 2010/04/28 00:21:30 s: moz2-darwin10-slave18
http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1272490048.1272495617.15588.gz OS X 10.6.2 mozilla-central leak test build on 2010/04/28 14:27:28 s: moz2-darwin10-slave10
http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1272551384.1272559391.29507.gz OS X 10.6.2 mozilla-central leak test build on 2010/04/29 07:29:44 s: moz2-darwin10-slave18
http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1272601426.1272606921.32151.gz OS X 10.6.2 mozilla-central leak test build on 2010/04/29 21:23:46
http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1272627138.1272634671.32043.gz OS X 10.6.2 mozilla-central leak test build on 2010/04/30 04:32:18 s: moz2-darwin10-slave22 http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1272611453.1272616336.24932.gz OS X 10.6.2 mozilla-central leak test build on 2010/04/30 00:10:53 s: moz2-darwin10-slave22
Priority: -- → P3
Whiteboard: [orange] → [orange][10.6]
http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1272745539.1272755257.28014.gz OS X 10.6.2 mozilla-central leak test build on 2010/05/01 13:25:39
http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1272830922.1272837219.16742.gz OS X 10.6.2 mozilla-central leak test build on 2010/05/02 13:08:42 s: moz2-darwin10-slave22
http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1272994048.1272999648.5347.gz OS X 10.6.2 mozilla-central leak test build on 2010/05/04 10:27:28 s: moz2-darwin10-slave22
http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1273076174.1273085092.9046.gz OS X 10.6.2 mozilla-central leak test build on 2010/05/05 09:16:14 s: moz2-darwin10-slave22
http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1273084222.1273093054.21731.gz OS X 10.6.2 mozilla-central leak test build on 2010/05/05 11:30:22 s: moz2-darwin10-slave10
http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1273088987.1273096613.7958.gz OS X 10.6.2 mozilla-central leak test build on 2010/05/05 12:49:47 s: moz2-darwin10-slave18
http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1273104362.1273109033.20389.gz OS X 10.6.2 mozilla-central leak test build on 2010/05/05 17:06:02 s: moz2-darwin10-slave18
http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1273160661.1273166182.20776.gz OS X 10.6.2 mozilla-central leak test build on 2010/05/06 08:44:21 s: moz2-darwin10-slave09
http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1273152331.1273159965.28126.gz OS X 10.6.2 mozilla-central leak test build on 2010/05/06 06:25:31 s: moz2-darwin10-slave09
http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1273134004.1273142922.26616.gz OS X 10.6.2 mozilla-central leak test build on 2010/05/06 01:20:04 s: moz2-darwin10-slave09
http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1273171001.1273175909.27077.gz OS X 10.6.2 mozilla-central leak test build on 2010/05/06 11:36:41 s: moz2-darwin10-slave22
http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1273188054.1273192838.27419.gz OS X 10.6.2 mozilla-central leak test build on 2010/05/06 16:20:54 s: moz2-darwin10-slave18
http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1273207954.1273213117.25455.gz OS X 10.6.2 mozilla-central leak test build on 2010/05/06 21:52:34 s: moz2-darwin10-slave18
http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1273330387.1273335052.9910.gz OS X 10.6.2 mozilla-central leak test build on 2010/05/08 07:53:07 s: moz2-darwin10-slave18 http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1273320270.1273324946.10582.gz OS X 10.6.2 mozilla-central leak test build on 2010/05/08 05:04:30 s: moz2-darwin10-slave18
http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1273522579.1273532292.18172.gz OS X 10.6.2 mozilla-central leak test build on 2010/05/10 13:16:19 s: moz2-darwin10-slave18
http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1273526177.1273535136.27202.gz OS X 10.6.2 mozilla-central leak test build on 2010/05/10 14:16:17 s: moz2-darwin10-slave24
http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1273530094.1273539618.9286.gz OS X 10.6.2 mozilla-central leak test build on 2010/05/10 15:21:34 s: moz2-darwin10-slave16
http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1273533642.1273542749.20066.gz OS X 10.6.2 mozilla-central leak test build on 2010/05/10 16:20:42 s: moz2-darwin10-slave18
http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1273537541.1273546685.538.gz OS X 10.6.2 mozilla-central leak test build on 2010/05/10 17:25:41 s: moz2-darwin10-slave24
http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1273542613.1273551979.19687.gz OS X 10.6.2 mozilla-central leak test build on 2010/05/10 18:50:13 s: moz2-darwin10-slave20
http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1273546274.1273553697.25740.gz OS X 10.6.2 mozilla-central leak test build on 2010/05/10 19:51:14 s: moz2-darwin10-slave16
http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1273556204.1273561575.21803.gz OS X 10.6.2 mozilla-central leak test build on 2010/05/10 22:36:44 s: moz2-darwin10-slave16
http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1273588928.1273596071.23773.gz OS X 10.6.2 mozilla-central leak test build on 2010/05/11 07:42:08 s: moz2-darwin10-slave24
http://tinderbox.mozilla.org/Firefox/1273591346.1273599525.4482.gz OS X 10.6.2 mozilla-central leak test build on 2010/05/11 08:22:26 s: moz2-darwin10-slave18
http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1273593926.1273600658.8719.gz OS X 10.6.2 mozilla-central leak test build on 2010/05/11 09:05:26 s: moz2-darwin10-slave16
http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1273600959.1273608066.3433.gz OS X 10.6.2 mozilla-central leak test build on 2010/05/11 11:02:39 s: moz2-darwin10-slave16
http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1273659986.1273667324.26533.gz OS X 10.6.2 mozilla-central leak test build on 2010/05/12 03:26:26 s: moz2-darwin10-slave24
http://tinderbox.mozilla.org/showlog.cgi?log=Firefo/1273804873.1273810519.21804.gz OS X 10.6.2 mozilla-central leak test build on 2010/05/13 19:41:13
http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1274007590.1274012799.24397.gz OS X 10.6.2 mozilla-central leak test build on 2010/05/16 03:59:50 s: moz2-darwin10-slave24
http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1274041860.1274046542.31841.gz OS X 10.6.2 mozilla-central leak test build on 2010/05/16 13:31:00
http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1274091894.1274097723.7510.gz OS X 10.6.2 mozilla-central leak test build on 2010/05/17 03:24:54
http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1274134396.1274139997.22204.gz&fulltext=1 OS X 10.6.2 mozilla-central leak test build [testfailed] Started 15:13, finished 16:47, took 94mins
Summary: macosx64-debug - intermittent timeout loading bloatcycle.html → macosx64-debug - intermittent timeout loading bloatcycle.html (leaktest)
Assignee: nobody → bear
Running bloattest in a loop to see if it's memory constrained. I'll let it run for a couple of days and will be monitoring the log for errors.
I was working on a test environment to see if memory (swapped or not) would help reduce the frequency of these errors but I was not able to run the test in a loop because I kept running into bug 449408 It wouldn't happen at the same spot, but it always happened within 10 iterations of the loop and then the computer would reboot.
http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1275638221.1275642904.28595.gz OS X 10.6.2 mozilla-central leak test build on 2010/06/04 00:57:01
http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1276072504.1276079118.31575.gz OS X 10.6.2 mozilla-central leak test build on 2010/06/09 01:35:04
http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1276073770.1276081111.10256.gz OS X 10.6.2 mozilla-central leak test build on 2010/06/09 01:56:10
the LAST Firefox 11 Builds hit this. (And the last 10, otherwise passing try builds too) I'm not so sure we can call it intermittent at this point.
(In reply to comment #239) > the LAST Firefox 11 Builds hit this. (And the last 10, otherwise passing try > builds too) > > I'm not so sure we can call it intermittent at this point. Do we disable this test then? bear indicates in comment #166 that he was hitting a kernel panic when trying to debug this.
Has the frequency really changed, or does it just look worse because bug 571443 is now eating all the otherwise-green runs?
To answer my question (to the extent I'm willing to), the last 72 hours have been 78% orange from this, 22% red from bug 571443 and burning pushes, 72 hours four weeks ago were 60% orange, 12.5% red, 27.5% green.
Just in case it's not clear: I got bored with pasting log links here, but it's still failing this way 80% of the time, and failing the bug 571443 way the other 20%. If we ever see another green, I'll paste the log link for that instead.
Josh, if we can get you one of these minis, can you take a look at this?
Email me and lets see if I can find a way to debug this remotely before getting a machine to me.
moz2-darwin10-slave21 set aside for this.
Assignee: bear → joshmoz
The problem is that --trace-malloc is pegging the CPU at 100%, everything runs slowly that the 3600 sec timeout is usually exceeded. Drop the --trace-malloc option and things run just fine.
I filed bug 573631 about this problem. I can repro locally on a Mac Pro.
Depends on: 573631
Assignee: joshmoz → nobody
Can we disable this test for now until the bug is fixed?
Assignee: nobody → catlee
http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1277331264.1277339153.23565.gz OS X 10.6.2 mozilla-central leak test build on 2010/06/23 15:14:24 s: moz2-darwin10-slave21
Yes, please disable this test for now on Mac64, with a comment pointing at bug 573631.
In this rare green log for Bd Mac64, the relevant elapsedTime=822.149264 http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1277868202.1277869718.31847.gz&fulltext=1 In the corresponding green log for Bd Mac32, elapsedTime=47.264345 http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1277868217.1277870426.2793.gz&fulltext=1 This is consistent with Josh's observation that trace-malloc is very slow on Mac64 (bug 573631).
Attachment #457842 - Flags: review?(bhearsum)
Comment on attachment 457842 [details] [diff] [review] Disable tracemalloc tests on osx64 Looks fine to me
Attachment #457842 - Flags: review?(bhearsum) → review+
Blocks: 579092
Comment on attachment 457842 [details] [diff] [review] Disable tracemalloc tests on osx64 changeset: 833:3856e7e15e31
Attachment #457842 - Flags: checked-in+
Fixed by disabling tracemalloc on osx64
Status: NEW → RESOLVED
Closed: 15 years ago
Resolution: --- → FIXED
Whiteboard: [orange][10.6] → [10.6]
Product: mozilla.org → Release Engineering
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: