Radu has been working with microsoft and citrix. We have found an actual fix,
although there is still no determination, officially, as to the cause. It
does appear to be load related. I have 2 servers that are not used much and
never had a problem, but the ones that run all apps crash.
Radu has been able to repair on the fly with the #pragma recover addition to
the citrix .mof files, and the script. I have had to do a repair on citrix
presentation server. I am providing the info below:
Steps that work:
1. Open %Sytemdrive%:\Program Files\Citrix\System32\Citrix\WMI and edit
all the *.mof files in this location (there should be 5). Edit with notepad
and add the following tag:
#pragma autorecover
at the beginning (above 1st line #pragma namespace) of each files. Pragma
autorecover will the add the MOF file to the list of files compiled during
repository recovery.
2. Repair Presentation Server (ignore errors)
3. Reboot
4. create a script of below and run it (Might need to grab SC.exe of the
reskits):
sc config winmgmt start= disabled
sc stop winmgmt
sleep 5
rd /s /q "C:\windows\system32\wbem\repository"
sc config winmgmt start= auto
sc start winmgmt
sleep 5
CD %windir%\system32\wbem
regsvr32 /s %systemroot%\system32\scecle.dll
regsvr32 /s %systemroot%\system32\userenv.dll
regsvr32 /s %systemroot%\system32\tscfgwmi.dll
mofcomp cimwin32.mof
mofcomp cimwin32.mfl
mofcomp rsop.mof
mofcomp rsop.mfl
for /f %%s in ('dir /b *.mof *.mfl') do mofcomp %%s
for /f %%s in ('dir /b *.dll') do regsvr32 /s %%s
wmiprvse /regserver
winmgmt /clearadap
winmgmt /regserver
winmgmt /resyncperf
CD C:\Program Files\Citrix\System32\Citrix\WMI
for /f %%s in ('dir /b *.mof *.mfl') do mofcomp %%s
Post by A MayersonThis is not a hardware problem. I've encountered the WMI issues on Dell, HP,
and IBM hardware. Question for you all, Are you running Microsoft Operations
Monitor or Edgesight against your farms? We experience some WMI issues prior
but not until recently have we encountered full on service hangs and crashes.
This coincides with our deployment of Edgesight 5.0 SP2. It may be just
coincidence, or it may be an external application is stepping on Citrix.
Post by Karon WActually - it may have been the upgrade, not the hardware. I started with
Dell PE 2850 servers running PS4.0, and it crashed there too. I don't think
it is hardware specific for that reason. I do think it is a disconnect btwn
citrix wmi and microsoft wmi. I went from citrix 1.8 to 4.0 in one giant
leap, so I haven't a clue what happened in btwn them.
I only have 10 servers (one is test) so I am not in nearly the straits you
are. Even so, the prospect of reloading from scratch is not a good one, and
that is the only thing I have found that will fix it - at least for a while -
until it crashes again. Not a workable solution by any means.
Post by CroitoruThank you for your info!
I'll send the script on Monday when I get back to the office (I'm located in
New Zealand) - will also raise a call with IBM and Citrix. Would be
interested to compare IBM drivers version, firmware and backplane firmware.
My rumour mill contacts say they've also seen it on IBM blades in other
environments, BUT it's not really confirmed.
I've been working on and off on this since last April, and peope before me
seem to thing it started when they moved the farm to IBM blades and upgraded
to 4.0 (the release was done at the same time).
Microsoft has been very helful with this, and we've tried a lot of Windows
"under the hood" approaches to detect and capture a process dump when WMI
breaks, but have been unable to pinpoint what is happening.
I have only seen another post on a Citrix forum about a situation with
"looping" namespaces (our ones are not just Citrix), but seemed to be a
one-off and the engineer repaired it by copying a good repository from
another machine (which is not really a good idea if you can actually rebuild
your own).
Again, thanks for the info, I will send through the scripts and my notes, as
Post by Karon WWe do NOT have a case open with microsoft. However, we do have a support
contract with MTM. However since this is such a time consuming issue, I am
wary to start with them and blow all my hours.
Please do send the script and anything else you thing may help!! I have been
researching for about 3 months (since this farm started dropping citrix wmi)
and it is making me nuts. I had researched before on the 4.0, but stopped
once we went to new servers and PS4.5/5.0.
we are using IBM Blade HS21's - W2K3 sp2 - citrix 4.5 (5.0) RO2 plus about 4
patches. upgraded microsoft WMI - no help.
I look forward to seeing what you have and what microsoft is telling you.
Thanks a WHOLE BUNCH!!
Good morning
I've been researching this for quite a few months. I can provide you with a
scripted "wmi recovery" action, which adjusts DCOM permissions and settings,
WMI namespace and rebuilds the repository (will do this on Monday).
Do you have a case open with Microsoft by any chance? We do, and I was
wondering if they could coordinate the effort.
So we can repair the corruption in most instances,but we do not know what
the root cause is and how to stop it. AS a result, the 100+ Citrix farm has
some of the advanced Citrix functionality cripled, as we get 2-5 server
breaking WMI every week.
I will compile a full list of our finding so far if you like, and see what
is similar/different between our environmets. Microsoft is unsure what the
problem could be at this stage.
But in a nutsell, we are using IBM Blades (HS20/21), Ibm directory for
hardware monitoring, Windows 2003 SP2 + WMI stability patch (backport
functionalisty from Vista/2008), PS 4.5.
Post by Karon WHey Croitoru - I tried the "admin user account". No go. I have tried the
recompile and got the following for ALL the files to recompile - included the
C:\Program Files\Citrix\System32\Citrix\WMI>for /f %s in ('dir /b *.mof
*.mfl')
do mofcomp %s
C:\Program Files\Citrix\System32\Citrix\WMI>mofcomp citrix.XP10FR3.mof
Microsoft (R) 32-bit MOF Compiler Version 5.2.3790.3959
Copyright (c) Microsoft Corp. 1997-2001. All rights reserved.
Parsing MOF file: citrix.XP10FR3.mof
MOF file has been successfully parsed
Storing data in the repository...
An error occurred while opening the namespace for object 2 defined on lines
23 -
0X80041002 Class, instance, or property 'Name' was not found.
Compiler returned error 0x80041002
This is the same error you see when you try to view server health from
Citrix Access Management Console.
Post by CroitoruHi Karon
We have a similar problem - are you using IBM hardware by any chance?
Have you found the issue? We have rebuild the servers and repositories and
it's still happening ... would like to work with you to find similarities
between the environments.
Would appreciate your response ...
Many thanks
Radu