Skip to main content
GameDev.net gamedev.net
🔒 Locked

Dualcore & Timing

Started by Kitt3n Oct 26, 2005 at 10:05 AM 9 replies 3.4k views
Original Post
Kitt3n
Kitt3n
Hi, Up until now, I was using the "rdtsc" instruction to measure my time-delta's. Now my colleague got a dualcore machine and it's giving problems because the delta-t's are very irregular (app runs 'skipping' frames). If you set the app to run only on cpu 1, then the problems immediately disappears. I found two solutions for the problem: 1 Run a thread with a Sleep (1), an realtime priority, with a threadAffinityMask to a certain processor, and update a 'currentTime' value (eg with rdtsc). Running on normal priority doesn't work correctly 2. Use the Time timeGetTime() (only 1 ms accuracy) I'm not really fond of the thread-idea, looks like overkill just to get a simple time - and I'm not sure I like the winmm.lib I need, just for the timeGetTime function. So is it possible to do whatever timeGetTime does myself? What does this function do that avoids the multi-cpu timing problems? Does spawn a thread - like I did - and thus limit the accuracy to 1 ms? Is it at all impossible to use the rdtsc instruction on dualcore machines? Thanks in Advance, Roger
visit my website at www.kalmiya.com
LilBudyWizer
LilBudyWizer
Sounds like which core the RDTSC executes on matters. So the processor affinity alone should be enough. Rather than polling though just have that thread execute the instruction as needed. Have it set there waiting for a signal. When it gets the signal, read it, store it, signal it stored it and go back to waiting.
Keys to success: Ability, ambition and opportunity.
nmi
nmi
Using the affinity mask is the correct choice. But why do you want to create an additional thread ? Just set the mask for the thread that wants to execute rdtsc. If you have multiple threads, you can attach each to another processor, so calling rdtsc will execute correct values per thread.
Extrarius
Extrarius
I believe the problem also exists with QueryPerformanceCounter, but that MS was going to fix it soon (or already has). Maybe you should look into using this function instead..? It performs basically the same function as 'rdtsc', but it's slower (takes more time to execute, but that shouldn't matter if you call it once a frame), it gives you the frequency (QueryPerformanceFrequency), and you can let MS worry about such things =-P
"Walk not the trodden path, for it has borne it's burden." -John, Flying Monk
Kitt3n
Kitt3n

>Rather than polling though just have that thread execute the instruction as >needed. Have it set there waiting for a signal. When it gets the signal, read >it, store it, signal it stored it and go back to waiting.
That's an idea - at least that way I get rid of the ugly leeching cpu-time.
I'm curious if the timer-resolution will be high enough..

>But why do you want to create an additional thread ?
Because then I don't have to do 'special things' for my little timer-class
to work. I want it to be simple - just get a time-value - how it works internally shouldn't matter to client code.

>If you have multiple threads, you can attach each to another processor, so
>calling rdtsc will execute correct values per thread.
I'd rather let the OS handle processor affinity for my threads (currently
got 8 or so).

>I believe the problem also exists with QueryPerformanceCounter, but that MS was going to fix it soon (or already has).
I also thought that QueryPerformanceCounter had the same problem, so
that's why I didn't try it actually... but I'm going to check it, if
it work's I'll take it (no threads needed, no winmm.lib needed & accurate)

Still I'm curiuous what happens internally in TimeGetTime, or a fixed
QueryPerformanceCounter and how they handle multi-processor machines.
...or what does Linux do for that matter?




visit my website at www.kalmiya.com
Jan Wassenberg
Jan Wassenberg
See article for background.
Update:
	// CPU Timestamp Counter (incremented every clock)	// ns resolution, moderate precision (poor clock crystal?)	//	// issues:	// - multiprocessor systems: may be inconsistent across CPUs.	//   we could discard really bad values, but that's still inaccurate.	//   having a high-priority thread with set CPU affinity read the TSC	//   might work, but would be rather slow. could fix the problem by	//   keeping per-CPU timer state (freq and delta). we'd use the APIC ID	//   (cpuid, function 1) or GetCurrentProcessorNumber (only available	//   on Win Server 2003) to determine the CPU. however, this is	//   too much work for little benefit ATM, so call it unsafe.	// - deep sleep modes: TSC may not be advanced.	//   not a problem though, because if the TSC is disabled, the CPU	//   isn't doing any other work, either.	// - SpeedStep/'gearshift' CPUs: frequency may change.	//   this happens on notebooks now, but eventually desktop systems	//   will do this as well (if not to save power, for heat reasons).	//   frequency changes are too often and drastic to correct,	//   and we don't want to mess with the system power settings => unsafe.


Conclusion: RDTSC is not a safe-for-general-use time reference. Here's what I'd do:
1) if RDTSC is "safe" on this computer, use it because it's very high res and low overhead.
2) otherwise check QueryPerformanceFrequency; if it's not similar to CPU clock, assume it is using a safe timesource (working around the issues described in article).
3) Otherwise, you are screwed and first have only a 1ms timer (TimeGetTime). If that's not enough, what you can do is either: implement the "per-CPU timer state" thing mentioned above OR write a kmode driver that accesses the PIT directly (ugh; if someone else has this idea, boom).
E8 17 00 42 CE DC D2 DC E4 EA C4 40 CA DA C2 D8 CC 40 CA D0 E8 40E0 CA CA 96 5B B0 16 50 D7 D4 02 B2 02 86 E2 CD 21 58 48 79 F2 C3
Extrarius
Extrarius
Quote:
Original post by Kitt3n
[...]Still I'm curiuous what happens internally in TimeGetTime, or a fixed
QueryPerformanceCounter and how they handle multi-processor machines.
...or what does Linux do for that matter?
Well, I opened up winmm.dll in a dependancy viewer and I see:
NtSetTimerResolution
NtQueryTimerResolution
NtSetTimer
NtCancelTimer
NtCreateTimer
from NTDLL.dll

and also to QueryPerformanceCounter, so who knows what timeGetTime does.
"Walk not the trodden path, for it has borne it's burden." -John, Flying Monk
Kitt3n
Kitt3n
>If that's not enough, what you can do is either: implement the "per-CPU timer
>state" thing mentioned above OR write a kmode driver that accesses the PIT >directly
How would that work in general?

I guess I'd need a reference to what time the other processor is running
on... So on init, I (somehow) get the current time for all processors, eg:
CpuA: rdtsc= 10 (let's just assume some abritrary value)
CpuB: rdtsc=500

Now assume I want to profile a piece of code - so I call rdtsc and it
happens to run on CpuB and returns 510. I call the rdtsc again,
this time it happens to run on CpuA and returns 25.

So the delta would be (25-10) - (510-500) = 5

... is this the way to go, or am I completely messing up things now? :)

/R

visit my website at www.kalmiya.com
Jan Wassenberg
Jan Wassenberg
Exactly! For increased accuracy, you may also want to account for long-term drift by re-syncing periodically.
The way to get TSC values on all CPUs is by cycling through CPU affinity masks. You need such code anyway to detect hyperthreading, which tells you if you're dealing with a real SMP system (which will have the above problems) or just 2 units.
E8 17 00 42 CE DC D2 DC E4 EA C4 40 CA DA C2 D8 CC 40 CA D0 E8 40E0 CA CA 96 5B B0 16 50 D7 D4 02 B2 02 86 E2 CD 21 58 48 79 F2 C3
Kitt3n
Kitt3n
>Exactly! For increased accuracy, you may also want to account for long-term
>drift by re-syncing periodically.
>The way to get TSC values on all CPUs is by cycling through CPU affinity

I implemented a first version of this in the meanwhile, and added
compensation for the time needed for threadswitching when I'm cycling
through all cpu's getting the rdtsc.

Anyway, seems like it's working on the dualcore amd64 I've got here.

Now to find a intel-dualcore-HT machine - to see if the logicalCPU
thing will also work (I might still have a bug in the apic-function
where you need to seperate the bits to get physical and logical cpu's).

Regards,
Roger

visit my website at www.kalmiya.com
Jan Wassenberg
Jan Wassenberg
Quote:
I implemented a first version of this in the meanwhile, and added
compensation for the time needed for threadswitching when I'm cycling
through all cpu's getting the rdtsc.
Anyway, seems like it's working on the dualcore amd64 I've got here.

Cool. Could you please post the code? I'd be interested in having a look.

Quote:
Now to find a intel-dualcore-HT machine - to see if the logicalCPU
thing will also work (I might still have a bug in the apic-function
where you need to seperate the bits to get physical and logical cpu's).

Unfortunately I don't have hardware to test either, but ain't no teammates (some of whom are known to have HT) complained about incorrect CPU info yet :)
Here's my code:
static uint log_id_bits;	// bit index; divides APIC ID into log and physstatic const uint INVALID_ID = ~0u;static uint last_phys_id = INVALID_ID, last_log_id = INVALID_ID;static uint phys_ids = 0, log_ids = 0;// count # distinct physical and logical APIC IDs for get_cpu_count.// called on each OS-visible "CPU" by on_each_cpu.static void count_ids(){	// get APIC id	u32 regs[4];	if(!ia32_cpuid(1, regs))		debug_warn("cpuid 1 failed");	const uint id = bits(regs[EBX], 24, 31);	// partition into physical and logical ID	const uint phys_id = bits(id, 0, log_id_bits-1);	const uint log_id  = bits(id, log_id_bits, 7);	// note: APIC IDs are assigned sequentially, so we compare against the	// last one encountered.	if(last_phys_id != INVALID_ID && last_phys_id != phys_id)		cpus++;	if(last_log_id  != INVALID_ID && last_log_id  != log_id )		cpus++;	last_phys_id = phys_id;	last_log_id  = log_id;}// fix CPU count reported by OS (incorrect if HT active or multicore);// also separates it into cpu_ht_units and cpu_cores.static void get_cpu_count(){	debug_assert(cpus > 0 && "must know # 'CPU's (call OS-specific detect first)");	// get # "logical CPUs" per package (uniform over all packages).	// TFM is unclear but seems to imply this includes HT units *and* cores!	u32 regs[4];	if(!ia32_cpuid(1, regs))		debug_warn("ia32_cpuid(1) failed");	const uint log_cpu_per_package = bits(regs[EBX], 16, 23);	// .. and # cores	if(ia32_cpuid(4, regs))		cpu_cores = bits(regs[EBX], 26, 31)+1;	else		cpu_cores = 1;	// if HT is active (enabled in BIOS and OS), we have a problem:	// OSes (Windows at least) report # CPUs as packages * cores * HT_units.	// there is no direct way to determine if HT is actually enabled,	// so if it is supported, we have to examine all APIC IDs and	// figure out what kind of "CPU" each one is. *sigh*	//	// note: we don't check if it's Intel and P4 or above - HT may be	// supported on other CPUs in future. all processors should set this	// feature bit correctly, so it's not a problem.	if(ia32_cap(HT))	{		log_id_bits = log2(log_cpu_per_package);	// see above		last_phys_id = last_log_id = INVALID_ID;		phys_ids = log_ids = 0;		if(on_each_cpu(count_ids) == 0)		{			cpus         = phys_ids;			cpu_ht_units = log_ids / cpu_cores;			return;	// this is authoritative		}		// OS apparently doesn't support CPU affinity.		// HT might be disabled, but return # units anyway.		else			cpu_ht_units = log_cpu_per_package / cpu_cores;	}	// not HT-capable; return 1 to allow total = cpus * HT_units * cores.	else		cpu_ht_units = 1;	cpus /= cpu_cores;}
E8 17 00 42 CE DC D2 DC E4 EA C4 40 CA DA C2 D8 CC 40 CA D0 E8 40E0 CA CA 96 5B B0 16 50 D7 D4 02 B2 02 86 E2 CD 21 58 48 79 F2 C3

Topic Locked

This topic has been locked by a moderator. New replies are not allowed.

Sign in to reply to this topic.