I have been using PDStorage for various reasons in the past, but only recently learned that the timeout on PDStorage values is reset on activity (including read), not just write of the values. I learnt that here:
We have a cluster of 11 appliances in our primary sync group. To ensure consistency when editing a particular PDStorage value (list of iprange), we have the F5 load balancers set to send all requests for a particular URL to a particular appliance. However we are still noticing that the values are being overwritten with old data.
What I think is happening is that, while the webpage we access to control the values of the PDStorage always hits a single appliance, users that hit a rule that READ the value could be on any appliance. When the value is read, the timeout on the value is reset, and that appliance then takes ownership of the PDStorage object and at the next sync that copy is distributed to all other appliances.
Can anyone advise whether my theory is accurate? And if so, is there any way around it? In reality, we would like the timeout on the values to depend on the last time they were written, not read. Are there any controls to allow for that? Is it an option to set the sync time to zero, in an attempt to get the sync to happen instantly across the cluster? What kind of extra load would that put on the boxes? We have 50,000 devices accessing this cluster, including probably 4,000+ PDStorage values.
As some background, this is the particular project I'm working on right now, which I have mentioned in another post:
I have created a room management system to allow staff to turn Internet and Social Networking on/off on a room by room basis. There are currently around 230 rooms in it. I used:
1 List of String containing a list of comma-separate information about Site, Room, and Subnet, eg Campus1, Room2, 192.168.7.0-192.168.7.127
2 lists of IP ranges (one for Internet, one for Social Networking sites), which contain a list of subnets to block by default
2 pdstorage lists of IP ranges which will flip the default value of the block to it's other value if an entry exists
2 block pages, one to show a lit of all campuses, and one to show the rooms within a selected campus
2 block pages telling users what is blocked and who to talk to about getting it changed
The block rule is something like:
if (client IP is in a range in list defaultblock and client IP is not in range in list defaultoverride) or (client IP is not in range in list defaultblock and client IP is in range in list defaultoverride) then block.
It all seems to be working great, until we put it in production and have multiple appliances reading the PDStorage values.
Any comments or suggestions would be most welcome.
Is anyone from McAfee able to comment on my theory about PDStorage value "ownership"? It affects whether I need to migrate a legacy system or not.
I'm not McAfee, but you are right. Every PD Storage Variable contains also a timestamp about the last access.
If you use global variables they will by synced in your defined time interval (10minutes defaults).
The latest Variable access wins when the appliances selects the "right value".
We do not use the global PDs variable any longer. Since all Proxies has to exchange their hole PDs to any proxy in its group there might be a traffic peaks during the sync time. With 3 or 4 proxies we did not have any problems, but after syncing the storage over 12 proxies we saw a performance decrease and traffic peaks every 10 minutes.
Thanks for your helpful answer. It definitely matches up with what I'm seeing.
Are you able to share what you are using now that you have moved away from PDS, and if possible what you were using them for?